This week I'm finishing the Winter 2024 semester, which means scaling another mountain of marking. And much like climbing any great peak, the key to success is to wend your way instead of trying to move in straight line. Survival demands regular rest, and at this time of year, my colleagues and I start to talk about our "ABM projects": Anything-But-Marking is a time honoured tradition among professors in our department, started decades ago by one of my mentors, Evan Weaver. ABM is how you survive marking.
For the past year, my ABM project has been ChatCraft. It's become so important to me that in December I decided to make it the focus of my open source project course in the winter. After students learn the skills necessary to work in open source, the second of my two open source courses allows them to go deeper on a single open source project over 14 weeks. No quizzes, no labs, no assignments, no exam–just pull requests, code review, triage meetings, and weekly releases. It's never the same thing twice, which keeps me on my toes. Over the past 15 years I've used all kinds of different open source projects as our focus, many with Mozilla, sometimes in low-level languages, often involving web technologies, and now focusing on AI and LLMs via ChatCraft.
The first problem with getting students involved in working with LLMs is finding funding. It's not ridiculously expensive to talk with AI, but it does involve cost, and despite what you might imagine, it's almost impossible to get funding for something like this in the context or a course. You need "private investment" or an "industry partner" to help you. I was lucky enough to find that investor: my friend and project founder, Taras Glek. When I told Taras what I was thinking about doing, he immediately said he'd be willing to help fund and support the students' work. Thank you Taras!
Armed with an exciting open source project and credits for OpenAI and OpenRouter.ai, I welcomed a small team of eager students to the project and got them started researching technologies, looking at existing issues, and planning our first release.
Since then, we've spent the past three months doing weekly triage and planning meetings, reviewing each other's work, interacting with the growing community of users, and shipping almost weekly releases. There have been 10 releases in all, with Katie tagging v2.0.0 yesterday. You can see the how they worked and what they did each week in the course Submissions Page and read their blogs, which are full of interesting insights and learning.
Together we shipped some amazing new features this term. I'd like to talk about them in more detail in a follow-up to this post, but here are some highlights for me:
- Support for any OpenAI-compatible LLM provider, including easy switching between many providers and models
- Support of mathematical notation rendering
- Full Text-To-Speech and Speech-To-Text support (talk to ChatCraft or have ChatCraft talk to you!)
- Ability to download chat messages as text, image, or audio
- Support for image generation and multi-modal image support (share a screenshot or image as part of your chat)
- Support for WASM based language runtimes in browser via WASI (Python and Ruby), allowing developers to run code in the browser other than JS
- Provide free model support so you can use ChatCraft without an API Key
- RSS feed of a user's shared chats
- Addition of WebHandlers, allowing easy integration with your own backend function calls
- Many, many, many bug fixes!
Part of what made this term so much fun was that it was my first time not only allowing, but encouraging students to use AI when programming. You can't build ChatCraft effectively without also using it. Often dogfooding your open source project is a chore; but ChatCraft makes you so much more productive, it's easy to integrate it into your workflow. I use it every day.
I asked the students if they were allowed to use AI in their other courses and the consensus was that they weren't. It meant that during many of our meetings, not only would I teach them how to work on open source and code, but also show them new ways to use AI in their programming. For example, as they were learning how to do proper code reviews of each other's work, I showed them how to use the /import command to pull in a DIFF or patch from a GitHub PR and get ChatCraft to help explain the changes.
Slowly they started to add ChatCraft to their workflows, and over time they became really adept at extracting useful things from it. For example, in our final meeting I heard them tell stories about using it to refactor code ("I'd paste in the whole file plus my changes and start asking for help..."), debug a UI problem with a mix of screenshots and code, figure out TypeScript types for complicated changes, edit blog posts, understand what "most users expect," etc.
Part of what made ChatCraft pair so nicely with a course on opens source is that writing on GitHub Issues/PRs and writing in ChatCraft are so similar. I have found that the key is to talk to ChatCraft aspirationally, pretending that you're working with a colleague on a GitHub PR. The difference is that with ChatCraft, if you're clear enough on what you need, you can literally prompt your colleague to do the work.
Our efforts continued to be noticed, and the small community of passionate ChatCraft users has steadily grown. Every week there has been some new person to meet on Discord, a tweet of encouragement, or an email out of nowhere from someone using ChatCraft in an interesting new way:
This growth was made possible because of the energy that the students have poured into the project. New features take weeks to plan, implement, and then find and fix the bugs. You need significant mass to create a centre of gravity capable of drawing in new people, and the GitHub contribution graph for the winter term shows how much the students did:
As I was writing this, I was also reflecting on my own contributions. It's interesting for me to look at these graphs on a personal level, because it's clear exactly when my dad passed away in February, and the impact his death has had on me all term. This winter has also been a difficult time for me personally.
As much as the project benefited from the students' work, it also needed leadership. I was thankful for the reviews, testing, and guidance Taras gave this term. We spoke every few days about progress, evolving features, next priorities, and how to help the students succeed. Over the years doing this course, I've found that the most successful situations involve a bunch of junior developers being paired with someone senior. Having students only work with other students on a project team can't possibly produce the same outcomes or offer so rich an experience.
At our last meeting on Wednesday, Taras joined the call and had a chance to meet everyone and listen to the final presentations. We also had a chance to ask Taras questions about industry and his career. I thought he had some interesting things to say.
For example, he spoke at length about the importance of exploring new technologies (like AI), but also how critical it is to bring stubbornness and depth to your work. He explained that the difference between a good programmer and a bad programmer isn't necessarily how much they know, but rather, how stubborn they are to get to the bottom of a problem. You can't hope to succeed if you aren't willing to go deep with your tech stack in an effort to understand and fix bugs. Taras shared how his own work has required him to move all the way from the browser down into the kernel. Doing so was only possible because he was willing to ask for help and reach out to experts along the way.
He went on to connect these ideas with your effectiveness at building a network of peers. Here he emphasized the role of professional networking beyond the superficial "connect on LinkedIn" style that so many people at the students' stage suggest. He encouraged them to build meaningful relationships with people they genuinely enjoy working with and who enjoy working with you. From his own experience, he suggested that one effective way to do this is to spend time in your youth in a tech-hub city, where these types of connections and opportunities to meet other smart people are easier to come by.
On the topic of open source, Taras told many stories about his own contributions, blogging, and other publicly-visible work leading to interesting personal opportunities and jobs that would have been hard to get in any other way. The students were able to share how their own blog posts have been received, with thousands of views and some even getting offers to come and work on similar AI products.
Overall, I'm really pleased with the choice to focus on ChatCraft this term, and thankful for the small community of people who joined me. Thanks to Taras for his generosity and to the students for their willingness to trust me when I said they were capable of doing this work, despite it looking nothing like the rest of their courses. I'll never forget the meeting where Yumei announced, "I'd like to try something hard for the next release..." (she followed it up by adding support for running Python in the browser). Unfortunately, I need to get back to marking, so I'll leave you with this benediction:
May you find Yumei's courage and try something hard in the next release.