Last week I wrote about creating a JsonConverter to handle some funky data conversion problems I was experiencing using System.Text.Json.JsonSerializer's Deserialize() method. I had some JSON-formatted data coming back from Azure DevOps and when I was parsing it, there were some data type formats that System.Text.Json.JsonSerializer wasn't especially happy about. The solution to the problem was to create an implementation of JsonConverter to handle the custom data conversion logic.

Of course, nothing in software ever goes smoothly so I ran into a weird problem along the way. I started getting an error that said System.Text.Json.JsonException: The converter 'EverythingToStringJsonConverter' read too much or not enough.

The Problem

The exception was happening when my JsonConverter implementation found a StartObject JsonTokenType in the JSON it was trying to parse.

At this point here's what my code looked like in the JsonConverter:

public class EverythingToStringJsonConverter : JsonConverter<string>
{
    public override string Read(ref Utf8JsonReader reader,
        Type typeToConvert,
        JsonSerializerOptions options)
    {

        if (reader.TokenType == JsonTokenType.String)
        {
            return reader.GetString() ?? String.Empty;
        }
        else if (reader.TokenType == JsonTokenType.Number)
        {
            var stringValue = reader.GetDouble();
            return stringValue.ToString();
        }
        else if (reader.TokenType == JsonTokenType.False ||
            reader.TokenType == JsonTokenType.True)
        {
            return reader.GetBoolean().ToString();
        }
        else
        {
            Console.WriteLine($"Unsupported token type: {reader.TokenType}");

            return "(BTW, something weird happened)";
        }
    }

    public override void Write(Utf8JsonWriter writer, string value, JsonSerializerOptions options)
    {
        writer.WriteStringValue(value);
    }
}

BTW, here's the full error message:

Unhandled exception. System.Text.Json.JsonException: The converter 'EverythingToStringJsonConverter' read too much or not enough. Path: $.value[0].fields['System.CreatedBy'].newValue | LineNumber: 64 | BytePositionInLine: 23.
   at System.Text.Json.ThrowHelper.ThrowJsonException_SerializationConverterRead(JsonConverter converter)
   at System.Text.Json.Serialization.JsonConverter`1.VerifyRead(JsonTokenType tokenType, Int32 depth, Int64 bytesConsumed, Boolean isValueConverter, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonDictionaryConverter`3.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TDictionary& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at Program.<<Main>$>g__ParseUsingTypedValues|0_0(String json) in /Users/benday/code/temp/Benday.JsonConverterSample/Benday.JsonConverterSample/Program.cs:line 12
   at Program.<Main>$(String[] args) in /Users/benday/code/temp/Benday.JsonConverterSample/Benday.JsonConverterSample/Program.cs:line 8

What's Happening? What am I doing wrong?

What the heck does "The converter read too much or not enough" even mean?!?!?! Well, it's important to remember that that Utf8JsonReader is basically behaving like a stream. It's got a point in the stream that it's looking at and it moves along in the stream depending on what you call.

I thought I was being super clever in my else block in my JsonConverter by eating any exceptions and returning a default value. I wasn't telling the reader what to do with itself and most importantly, I wasn't handling the StartObject JsonTokenType.

The Fix: Use reader.Skip() to Discard Values

The fix was to explicitly handle the StartObject token type and then to continue to throw exceptions if something else went wrong. Part of my problem was that for the code that I was writing, I didn't care about the object values that I was getting -- I only cared about the scalar values. I thought that I'd be able to just have everything be handled by that else block. Nope.

For the JsonTokenType.StartObject, I was basically discarding the value of that JSON property...but in order to do that correctly, I needed to tell the Utf8JsonReader that that was my intention by calling reader.Skip(). Once I called Skip(), then the reader was able to just move on and that was the end of the "too much or not enough" error.

Here's the updated Read() method:

public override string Read(ref Utf8JsonReader reader,
    Type typeToConvert,
    JsonSerializerOptions options)
{

    if (reader.TokenType == JsonTokenType.String)
    {
        return reader.GetString() ?? String.Empty;
    }
    else if (reader.TokenType == JsonTokenType.Number)
    {
        var stringValue = reader.GetDouble();
        return stringValue.ToString();
    }
    else if (reader.TokenType == JsonTokenType.False ||
        reader.TokenType == JsonTokenType.True)
    {
        return reader.GetBoolean().ToString();
    }
    else if (reader.TokenType == JsonTokenType.StartObject)
    {
        reader.Skip();
        return "(not supported)";
    }
    else
    {
        Console.WriteLine($"Unsupported token type: {reader.TokenType}");

        throw new System.Text.Json.JsonException();
    }
}

Summary

If you're running into the extremely confusing Utf8JsonReader "converter read too much or not enough" exception, you're probably -- well -- reading too much or not enough. ("Thanks, Ben. You're a genius." Not helpful? You're welcome. LOL.) In short, you're probably not keeping that reader up to date on what your intentions are when you're inside of your JsonConverter class. In this case, I needed to call reader.Skip().

Here's the full sample code for my JsonConverter.

I hope this helps.

-Ben