Easy Code Share > Python > Solving Post JSON to Python CGI FieldStorage Error

Solving Post JSON to Python CGI FieldStorage Error

Python CGI module gets popular among developers for only unstructured data, but its FieldStorage() gives rise to an error for posting JSON data. The article describes the problem in details and raise a solution.

You should configure Apache CGI on Windows or Linux before studying this topic, because practices make perfect.

All codes here are not complicated, so you can easily understand even though you are still students in school. To benefit your learning, we will provide you download link to a zip file thus you can get all source codes for future usage.

Estimated reading time: 5 minutes



Source Code Download

We have released it under the MIT license, so feel free to use it in your own project or your school homework.


Download Guideline

  • Prepare HTTP server such as XAMPP or WAMP in your windows environment.
  • Download and unzip into a folder that http server can access.


Python CGI FieldStorage for JSON

Most developers use Python’s module cgi to accept POST data which comes from Html clients. Unfortunately, for JSON data, it failed. Let us investigate the reason for errors in the section.


POST Data by Content-Type of TEXT

If you send both TEXT string str_data and JSON data json_data to Python WebApp that uses cgi.FieldStorage(), the server site can get only TEXT string. Note that dataType should be for TEXT, so that the header would include “Content-Type: text/html”.

var str_data = 'EXECUTE';
var json_data = {'commands': {'a00abf1ddb37': [{'devices': ['FAN'], 'execution': [{'params': {'on': true}, 'command': 'OnOff'}]}]}};

$("#btn1").on("click", function () {
    console.log("\nSEND TO cgi.FieldStorage => ");
        url: "webapp-fieldstorage.py",
        type: "POST",
        data : {'string': str_data, 'json': json_data},
        dataType: "text", // Default. This line can be ignored.
        success: function(r) {
            console.log("RECEIVE =>");

Here, we use a complex JSON data for interpretation.

Complex JSON Data

The server site, webapp-fieldstorage.py, retrieves TEXT string and the complex JSON data, separately. However, JSON data is None type.

{'string': 'EXECUTE', 'json': None}

For TEXT string, cgi.FieldStorage() did it correctly for ‘EXECUTE’.

import json, cgi

form = cgi.FieldStorage()

print('Content-Type: text/html\n')

string = form.getvalue('string')
payload = form.getvalue('json')
print({'string': string, 'json': payload})

key = 'json[commands][a00abf1ddb37][0][devices][]'


Why Mistakes Happen

We guess there is invalid data structure inside Python cgi.FieldStorage(), and find that using an abnormal key as below, you can get the desired value, FAN, out of JSON data.

key = 'json[commands][a00abf1ddb37][0][devices][]'

Look at the screen layout, clicking on the left button will test the cgi.FieldStorage() error which results in Python None type for JSON data. Also, you can find that cgi.FieldStorage() treat an abnormal string as key in its internal data structure. Indeed, even applying the wrong key we can get the real value, FAN.

CGI FieldStorage Retrieve JSON POST Data Error


The Solution to Entire JSON Retrieval

Alternatively, Python’s module sys can solve the problem. However, the request Content-Type should be JSON, rather than TEXT. We interpret it with a complex JSON, and retrieve one part of it to show that the approach will completely decode JSON data.


POST Data by Content-Type of JSON

At this time, we send both TEXT string str_data and JSON data json_data to another Python WebApp that uses module sys. Fortunately, the server site can get entire data, even complex JSON data.

Note that dataType should be for JSON, dataType: "json", so that the header would include “Content-Type: application/json”. In addition, what you send should be made by JSON.stringify().

$("#btn2").on("click", function () {
    console.log("\nSEND TO sys.stdin => ");
        url: "webapp-stdin.py",
        type: "POST",
        data : JSON.stringify({'string': str_data, 'json': json_data}),
        dataType: "json",  // HTTP Header will be modified for JSON type automatically.
        success: function(r) {
            console.log("RECEIVE =>");
            r = JSON.stringify(r);

For the server part, webapp-stdin.py, let us inspect inside the JSON POST data and retrieve commands['a00abf1ddb37']. The action indicates that the received is of Python’s Dictionary type, rather than TEXT only. Finally, make commands['a00abf1ddb37'] a TEXT string and send it back to the client site with header protocol of “Content-Type: application/json”.

import json, sys

dict1 = json.load(sys.stdin)

print('Content-Type: application/json\n')
json_data = dict1['json']
commands = json_data['commands']


Truly Solution to JSON Data POST for Python WebApp

The screen layout below shows that response data are part of request data, and both of them are of JSON format, as indicated in browser’s console logs. Actually, this is a complete solution for Python WebApp to accept JSON POST data without mistakes.

SYS Stdin Inspect Complex JSON POST Data



This post not only reveals the error inside cgi.FieldStorage() data structure, but also give a prefect solution for JSON requests using a Python’s built-in module without installation. Eventually, the reason why you suffer such obstacles is clear.

Thank you for reading, and we have suggested more helpful articles here. If you want to share anything, please feel free to comment below. Good luck and happy coding!


Suggested Reading

Leave a Comment