The magic of working in open source

Ken McGrady -

Wondering what it’s like to work on the Streamlit open-source project? There are many reasons why we all love it. But the most exciting one is our focus on becoming the best tool in every data scientist’s toolchain.

Open source involves lots of stakeholders yet offers limited resources. So our biggest challenge is to prioritize and implement the most useful features.

In this post, you’ll learn:

  • How we prioritize new features
  • What goes into implementing new features
  • How we keep in touch with the community
  • How you can contribute to the open-source community

How we prioritize new features

We prioritize new features every quarter:

  • Our product team decides which features will evolve the product and its audience.
  • Our engineering team builds solutions to GitHub issues.
  • Our community team monitors our social channels and advocates for the community’s needs.

For example, these features took countless hours of brainstorming, prototyping, and testing: multipage apps, new caching primitives, camera input, and updated dataframes.

In between the larger features, we tackle small delightful experiences, fix bugs, improve built-in charts, and add parameters to our APIs (tooltips, gap sizes, disabled widgets, etc.).

What goes into implementing new features

Before we start working on a new feature, we talk to our Data Science team and Streamlit Creators. Together, we decide which feature has the right amount of complexity and the most intuitive API (though there’s rarely a single solution for everyone’s use case).

A feature typically starts out as a simple “couple-of-lines change” that grows into a discussion on how it’ll impact the users, how it could be misused, and if it’ll keep our software resilient. We sort through lots of community feedback before finally pulling the trigger.

Once we build and release the feature, we move forward super-fast by:

  • Unit-testing it to narrow down bugs in code;
  • End-to-end testing to test the full functionality of a feature;
  • And screenshot-testing it to make the visuals pixel-perfect.

From an engineering standpoint, we try to not break our API while keeping a semantic versioning promise. We work with our product and design teams to give our users the best experience by looking at the common data use cases and designing solutions that have room for change. Plus, all external contributors' code gets assigned a code reviewer. Often we assign two code reviewers because we’re not familiar with the context!

If you’re curious to learn more about how we implement new features, check out these posts:

How we keep in touch with the community

It can be a challenge for engineers to balance delivering features and talking to the community. We want to deliver our code on time, so our community conversations have a “context switching” tax. Our focus tends to be more on the quality of our product and less on the use cases, so our attention goes to the GitHub issues and bugs. We try to understand the issue, reliably reproduce it, and guesstimate its impact. Often, due to timing, we can’t fix the bug, but we get enough knowledge to help an external contributor solve the problem.

But we’re out there:

  • Our Engineering team posts release notes and responds to many posts on the forum.
  • Our Data Science team always has new ideas based on their Streamlit dogfooding.
  • Our Developer Relations team works with the community to produce rich content like 30 days of Streamlit.

We’re now part of Snowflake, and Snowflake’s mission is to mobilize the world’s data. Our community plays a big role in it. We believe in the full-employment theorem so we can always make Streamlit a better product for data scientists!

How you can contribute to the open-source community

Contributing to the open-source community is very rewarding. The software is free. And you can improve a single function or a whole discipline! But getting involved may seem daunting as most conversations are asynchronous. It takes time, patience, and fortitude.

If you want to get involved and help us make a stronger product, we’d love for you to do so! Here is how to get started:

Spend a month or two focused on the above—it’ll clarify for you how to help out in code. When ready, follow our contributing guidelines and take on a bug from our GitHub issues. Bugs are understandable, reproducible, and have a desired outcome. We identified some good first issues, but there are many more to choose from.

And finally, follow good software engineering practices in designing your solution and write tests (it saves the first comment in a code review). 🧑‍💻

Does this make you excited?

Want to work on open source as a job? Join our team! Our jobs require a unique skill set because Streamlit’s main value is delivering a clean and interactive user interface for developers, so we rely on strong frontend skills with TypeScript/React. And our developers interface with Streamlit using a simple Python API and server.

Here are our current job openings:

Thank you for being part of our community. If you have questions, please post them in the comments below, and you may see them answered in future blog posts. 😉

Happy coding! 🎈