Baldur Bjarnason

Web dev at the end of the world, from Hveragerði, Iceland

FormData and fetch, why is serialising a form such a pain?

The form is one of the basic building blocks of the web but so fraught with problems.

You have a form in your HTML.

You have hooked a function to its submit event. You can take care of the form submission directly and not have the entire page reload.

You have everything ready to handle whatever the server sends in response.

The form is well structured and accessible—nobody can accuse you of not paying attention to accessibility!

(Well, not again, at the very least. We all start off not paying attention to it because there is so much to pay attention to in web development. Then we learn and adjust.)

All of the building blocks are working and that’s when everything starts to go wrong.

  • You have problems serialising the entire form (with both known fields and future unknown fields) without pulling in a bespoke library and you’re already close to blowing your JS payload budget. (Nice form validation is a must and that’s where JS can make a huge difference.)
  • You could grab the values from the known inputs one by one, but this is a large form and it makes updating the form tedious.
  • It also means that your code is tied to this one form specifically and makes reusing it on all of the other forms tougher.
  • “All I get is an empty response with FormData”. You tried the ‘proper web standard’ solution, FormData, but you only got empty responses back.
  • Then you remember that you need to modify the data before it’s submitted and the last time you tried this (again, using FormData which clearly does not work) you ended up only posting the modified data to the server. The form’s data kept evaporating somehow. “The FormData is empty even though the text field has been filled in."
  • “FormData is only for file uploads, not regular forms."

You’ve reached a point where it’s tempting to just reach for a framework and a form building library, and just hope nobody notices that you’ve blown the JS payload budget to smithereens.

There is a solution to all of these problems, one that is universally supported by browsers and talks to every single kind of POST endpoint:

FormData.

—But, I told you. FormData doesn’t work.

It can work. Amazingly well, even. But the problems you (and me) keep having with it are very real.

As with everything else in the web stack, getting FormData to work is counter-intuitive. This is because web APIs are a mishmash of coding and architecture styles. Once you’ve figured one thing out, your mental model of, say querySelectorAll isn’t going to help you out with FormData.

FormData’s design is based on a series of assumptions and if you aren’t aware of them, using it is going to be fraught and painful.

The Content-Type

This is why you sometimes get back a 500 response or even just an empty response (I’m looking at you PHP) when using FormData to serialise a form.

When you submit FormData as the body of a fetch or XMLHttpRequest the content type is multipart/form-data, the same as you get when you’re uploading a file using a form.

A lot of servers just don’t handle that content type out of the box.

Because that content type is also used for file uploads, implementing it isn’t entirely straightforward.

And if the server-side is handled by a different department or team, getting them to add support is often a can of worms you don’t want to get into.

—See? I told you. I tried new FormData(form). This does not work.

The serialisation and data structure

FormData is neither a string (which is what you submit with a application/x-www-form-urlencoded content type) nor a regular object (which you would serialise with JSON.stringify and a application/json content type.

If you’re used to URL-encoded form bodies you might be tempted to add a value to the submission this way: formdata = formdata + "&updated=newvalue"; and then submit it as a URL-encoded body.

But this means that the server will only get your added updated field.

This is because you just submitted this: [object FormData]&updated=newvalue. Like I wrote above, turns out FormData isn’t a string.

If your server didn’t just throw an error it will have parsed this as:

{"[object FormData]": "", "updated": "newvalue"}

The same applies if you try this:

formdata.updated = "newvalue";
console.log(JSON.stringify(formdata));
// results in: {"updated":"newvalue"}

Because it isn’t a regular object either. It’s a collection of string key and string value entries.

You can manipulate it in a number of complex ways. But that’s for another day. Today, we’re going to stick with solving the initial problem we had in the simplest possible way.

—Right. I need to serialise this form, add a field, and then submit it to my possibly out-of-date server.

How to serialise a form, add a field, and then submit it to a possibly out-of-date server

We’re in the same position we were when we started. 1. We have the form in place. 2. The submit handler is hooked up and has a reference to the form. 3. Fetch is actually sending the request. We need to properly connect step 2 and 3.

So, without further ado:

const formdata = new FormData(form)
fetch('/test/thing', {
    method: 'POST',
    headers: {
        'Content-Type': 'application/x-www-form-urlencoded'
    },
    body: new URLSearchParams(formdata).toString()
}).then(result => {
// do something
}).catch(err => {
// fix something.
})

The magic trick here is that the URLSearchParams constructor, which is purpose-made for serialising objects to url-encoded strings, natively understands the entries-oriented data structure that FormData uses.

Turns out that “series of key-value entries serialised as a stream of [key, value] arrays” is a standard part of the JavaScript language. This means that a lot of APIs and classes have built-in methods for handling that structure.

In turn this means that, if your endpoint prefers JSON, you can use this serialisation method instead.

const formdata = new FormData(form)
fetch('/test/thing', {
    method: 'POST',
    headers: {
        'Content-Type': 'application/json'
    },
    body: JSON.stringify(Object.fromEntries(formdata.entries()))
}).then(result => {
// do something
}).catch(err => {
// fix something.
})

The only downside to using this method to serialise to JSON is that if your form has multiple inputs with the same name attribute then FormData creates multiple entries with the same key.

URLSearchParams serialises these directly and some server request body parsers automatically turn these structures into array values. This can be a useful convenience under some circumstances but isn’t universally supported. So, it might make sense for you to avoid this even if you aren’t serialising to JSON.

The JSON serialisation also doesn’t convert any of the values in any way. They’re all going to be strings. So, no booleans, nested objects, or integers. The url-encoded version has the same limitations but the server-side programmer knows that in advance whereas they may have different expectations of JSON bodies.

Add new URLSearchParams(formdata).toString() as a snippet to your preferred IDE. If you’re going to be working with forms, you’re going to need it.

I’m currently available for consulting and hire. If you need somebody to figure out how best to solve problems with standard web APIs and a minimal JS payload, get in touch.

You can also find me on Mastodon and Twitter