Build a Streamlit Form Generator app to avoid writing code by hand

Gerard Bentley -

Hey, community! 👋

My name is Gerard Bentley. I’m a backend engineer at Sensible Weather and a Streamlit Creator.

I made lots of Streamlit demos to understand our web API endpoints—that took writing a bunch of similar code. So instead of writing Python models and starter Streamlit code by hand, I combined Streamlit-Pydantic, Datamodel Code Generator, and the OpenAPI Specification and built the Streamlit Form Generator app.

In this post, I’ll show you how to build it step-by-step:

  • Get an OpenAPI Specification (OAS) from the user by upload, text box, or URL.
  • Generate Python code with Pydantic BaseModel classes from objects in the OAS API schema.
  • Create a ZIP archive of the generated code for users to download and make demos with.

Can’t wait to get started? Here's the app and the repo.

What is Streamlit Form Generator app?

The Streamlit Form Generator app makes code that accepts and validates user inputs according to an API spec. It will let you:

  • Get multiple user inputs of an API spec.
  • Parse it into Pydantic models.
  • Templatize a new Streamlit-Pydantic form-based app with user-selected options.

Here is the template generated by the example OAS:

In an internal dogfooding demo, I use this form input for making HTTP requests to my development server. I’ve included Streamlit-Folium to let users select the latitude and the longitude with a pin on a map.

Let’s skip writing similar Streamlit Forms and build this app together:

1. Get an OpenAPI Specification (OAS) from the user by upload, text box, or URL.

You can get user input for the app by doing the following:

  • Use an example OAS (from the Weather Insurance Quote API that I work on at Sensible Weather).
  • Upload a file containing an OAS.
  • Manually enter an OAS in a text box.
  • Fetch a file containing an OAS from a URL.

To start, install Streamlit. Use pip install streamlit in your Python environment of choice or see the detailed Get started guide for more options. After a conventional import streamlit as st, use a radio button to let the user choose their input method:

import streamlit as st

use_example = "Example OpenAPI Specification"
use_upload = "Upload an OpenAPI Specification"
use_text_input = "Enter OpenAPI Specification in Text Input"
use_url = "Fetch OpenAPI Specification from a URL"

st.header('OAS -> Pydantic -> Streamlit Form Code Generator')
input_method = st.radio(
    label="How will you select your API Spec",
    options=[use_example, use_upload, use_text_input, use_url],
)

st.subheader(input_method)

Your app will look something like this:

Change st.radio() to st.sidebar.radio() to see if the app looks better with this option in the sidebar. Or change it to st.selectbox() if you prefer the look of select boxes.

To get the raw OAS text, present the data at each step in an st.expander() for the user to inspect. The expander will hide the content if the user doesn't care:

from pathlib import Path

import httpx
from pydantic import BaseModel, HttpUrl
from streamlit.runtime.uploaded_file_manager import UploadedFile

# ...

@st.experimental_memo
def decode_uploaded_file(oas_file: UploadedFile) -> str:
    return oas_file.read().decode()

@st.experimental_memo
def decode_text_from_url(oas_url: str) -> str:
    try:
        response = httpx.get(oas_url, follow_redirects=True, timeout=10)
        return response.text
    except Exception as e:
        print(repr(e))
        return ""

class ValidURL(BaseModel):
    url: HttpUrl

def get_raw_oas(input_method: str) -> str:
    if input_method == use_example:
        st.write("This will demo how the app works!")
        oas_file = Path("quote-oas.json")
        raw_oas = oas_file.read_text()
    elif input_method == use_upload:
        st.write("This will let you use your own JSON or YAML OAS!")
        oas_file = st.file_uploader(
            label="Upload an OAS",
            type=["json", "yaml", "yml"],
            accept_multiple_files=False,
        )
        if oas_file is None:
            st.warning("Upload a file to continue!")
            st.stop()
        raw_oas = decode_uploaded_file(oas_file)
    elif input_method == use_text_input:
        st.write("This will parse raw text input into JSON or YAML OAS!")
        raw_oas = st.text_area(label="Enter OAS JSON or YAML text")
        if not len(raw_oas):
            st.warning("Enter OAS text to continue!")
            st.stop()
    elif input_method == use_url:
        st.write("This will fetch text from the URL containing a JSON or YAML OAS!")
        raw_oas_url = st.text_input(label="Enter the URL that hosts the OAS")
        try:
            oas_url = ValidURL(url=raw_oas_url)
        except Exception as e:
            print(repr(e))
            st.warning("Enter a valid HTTP(S) URL to continue!")
            st.stop()
        raw_oas = decode_text_from_url(oas_url.url)
    else:
        raise Exception("Unknown input_method")
    return raw_oas

raw_oas = get_raw_oas()
with st.expander("Show input OAS"):
    st.code(raw_oas)

Now your app should look something like this:

The point of this get_raw_oas block is to get the raw OAS text regardless of the input method:

  • In the "Example" branch, use the built-in Path class to read_text() from a file in the same folder as your Streamlit app (this works when the app is deployed to Streamlit Community Cloud and the file is in GitHub).
  • In the "Upload" branch, use st.file_uploader() to let the user drag and drop or choose their file. This provides None if the user hasn’t uploaded anything and a BytesIO-like object if they have.
  • In the "Text Input" branch, use st.text_area() to allow free text input. You can opt for something fancier such as Streamlit Quill for a rich text editor.
  • In the "URL" branch, let the user input URL as text, validate it with Pydantic's HttpURL class, and 'GET' the file from the URL using [httpx](<https://github.com/encode/httpx/>).
  • For the "Upload" and the "URL" branches, use st.experimental_memo to cache the results between re-runs. The "Example" branch runs fast enough, and the "Text Input" branch is already cached until changed (because that’s how Streamlit works).

In all of the user upload branches warn the user and stop the script if they haven't added anything. You can use this technique with Streamlit forms as well.

2. Generate Python code with Pydantic BaseModel classes from objects in the OAS API schema.

Now, let’s go over using Datamodel Code Generator to parse the raw input text into the Python Pydantic classes. It’s super convenient for making Python clients that communicate with APIs. You can do the following:

  • Get type validation and autocomplete.
  • Dump Pydantic models into the JSON format to call your API.
  • Parse the JSON response into Pydantic models for validation and further handling.

Dictionaries are more convenient for hacking around, but one misspelled key can make bug-hunting hard. Import the module and follow this documentation example of calling it a module. Use a TemporaryDirectory so as not to pollute the Streamlit Community Cloud app (you’ll allow downloading at the end).

If the OAS is very large, cache this function:

import ast
from dataclasses import dataclass
from pathlib import Path
from tempfile import TemporaryDirectory
from typing import List

from datamodel_code_generator import InputFileType, generate

# ...

@dataclass
class ModuleWithClasses:
    name: str
    code: str
    classes: List[str]

@st.experimental_memo()
def parse_into_modules(raw_oas: str) -> List[ModuleWithClasses]:
    with TemporaryDirectory() as temporary_directory_name:
        temporary_directory = Path(temporary_directory_name)
        module_files = generate_module_or_modules(raw_oas, temporary_directory)

        modules = []
        for module in module_files:
            module_code = module.read_text()

            module_ast = ast.parse(module_code)
            module_class_names = [
                x.name for x in module_ast.body if isinstance(x, ast.ClassDef)
            ]
            modules.append(
                ModuleWithClasses(
                    name=module.stem,
                    code=module_code,
                    classes=module_class_names,
                )
            )
    return modules

def generate_module_or_modules(raw_oas: str, output_directory: Path) -> List[Path]:
    output = Path(output_directory / "models.py")
    try:
        generate(
            raw_oas,
            input_file_type=InputFileType.OpenAPI,
            output=output,
        )
        return [output]
    except Exception as e:
        print(repr(e))
        try:
            generate(
                raw_oas,
                input_file_type=InputFileType.OpenAPI,
                output=output_directory,
            )
            return list(output_directory.iterdir())
        except Exception as e:
            print(repr(e))
            return []

modules = parse_into_modules(raw_oas)
if not len(modules):
    st.error("Couldn't find any models in the input!")
    st.stop()

st.success(f"Generated {len(modules)} module files")

all_module_models = []
for module in modules:
    import_name = module.name
    if import_name != "models":
        import_name = f"models.{import_name}"
    with st.expander(f"Show Generated Module Code: {import_name}"):
        st.code(module.code)

    for model_name in module.classes:
        all_module_models.append((module.name, model_name))

To let the user pick classes for Streamlit form inputs, use Python's Abstract Syntax Tree module to parse (ast.parse()) each generated module and grab them into a list.

Try the generate() function with an output target of a single file Path and a directory Path. Showing the full error text to the user might reveal sensitive data, so printing it to the console is good enough.

There are more full-featured code generators such as this open-source OpenAPITools project with over 30 languages. I used Datamodel Code Generator to make a flexible Streamlit + Pydantic template. It also produces idiomatic Pydantic code and has the flexibility to handle JSON schema, raw JSON/CSV/YAML data, and even a Python dictionary. (It’s a Python library, and I don't know how to run OpenAPI generator as an npm package or Java jar on Streamlit Community Cloud 😅.)

After you generate Pydantic models from the OAS, your app will look something like this:

3. Create a ZIP archive of the generated code for users to download and make demos with

Generate code for a Streamlit form with Streamlit-Pydantic and whatever generated classes the user selects as input possibilities.

Since most API specs will have some models that are only used within other models, let the user decide what they want to use for their Streamlit forms. You can further utilize the OpenAPI specification documentation to check what schema models are used as request bodies, but this will take a lot of modifications to the Datamodel Code Generator.

Use Python string manipulation to make a starting point for using the selected models in Streamlit. The guts are not very exciting and should be replaced by a templating library such as jinja2 if you want to make more complicated starter code:

if len(all_module_models) > 1:
    selections = st.multiselect(
        label="Select Models that will be Form Inputs",
        options=all_module_models,
        default=all_module_models[0],
        format_func=lambda x: f"{x[0]}.{x[1]}",
    )
else:
    selections = list(all_module_models)

def generate_header(models_with_modules: List[Tuple[str, str]]) -> str:
# ...

def generate_single_model_form(model: str) -> str:
# ...

def generate_multi_model_form(models: List[str]) -> str:
# ...

@st.experimental_memo
def generate_streamlit_code(selected_module_models: List[Tuple[str, str]]) -> str:
    streamlit_code = generate_header(selected_module_models)
    if len(selected_module_models) == 1:
        model_module, model = selected_module_models[0]
        streamlit_code += generate_single_model_form(model)
    else:
        models = [model for _, model in selected_module_models]
        streamlit_code += generate_multi_model_form(models)
    return streamlit_code

streamlit_code = generate_streamlit_code(selections)
with st.expander("Show Generated Streamlit App Code", True):
    st.code(body=streamlit_code, language="python")

One lacking aspect of templating is repetitive imports. Do a quick run of isort on the generated code to clean this up.

For the user to choose multiple models as form inputs, give the template a radio selector in the sidebar. You can adapt it to multi-page apps (I kept it in one page).

At this point, your app should look something like this:

Build the models.py if it’s contained in a single module, or build the models directory structure with all of the modules.

Streamlit's download_button works great with a bytes object:

@st.experimental_memo()
def zip_generated_code(modules: List[ModuleWithClasses], streamlit_code: str) -> bytes:
# ...

zip_bytes = zip_generated_code(modules, streamlit_code)
st.download_button(
    label="Download Zip of Generated Code",
    data=zip_bytes,
    file_name="generated_code.zip",
)

Wrapping up

Congratulations! You did it. You learned how to build your own Streamlit Form Generator app. I hope it will help you make user-friendly demos of your favorite APIs!

If you have any questions, please drop them below in the comments or reach out to me on GitHub, LinkedIn, or Twitter with your ideas or inspiring projects!

Happy Streamlit-ing! 🎈