Shipping Telescope 2.7.0

by David Humphrey

"shipping software is an endless job" – James Mai
"I watched in awe through screen-share" – Tim Roberts
"the train will leave the station, and there will be another train." – Amasia Nalbandian

This week I was trying to describe what online teaching is like, and it's hard to sum up in a few sentences.

Yesterday the students and I shipped our biggest release of the term, Telescope 2.7.0.  It included more than 80 bug fixes and new features by 30 contributors across +7K/-13K lines of code.  I'm especially impressed by the fact that they managed to remove nearly double what they added while still shipping the same features.

You can read about their work in their blog posts at https://telescope.cdot.systems/, but I'll summarize some of what I saw over the past two weeks.

What we shipped

I was looking back over the code we landed, and there's a lot.  Here are some highlights that stand out to me now:

  • Published a new custom eslint config package to use across our monorepo, which now gets auto-published to npmjs.org in CI/CD
  • A new project issue template, as well as better project organization
  • Two new microservices: dependency-discovery and Docusaurus
  • Integration between nginx, Supabase, Traefik, and our SSO Authorization microservice
  • Supabase database schemas for our user profiles and feeds
  • Retirement of our Users microservice
  • Lots of Docker, nginx, and Traefik fixes
  • pnpm fixes for our monorepo tooling
  • Reworking our Jest and ESLint tooling
  • Lots of new and reworked tests, including for Elasticsearch, Search, and SSO services
  • Updates to our Satellite microservice framework, which I hope will finally be moved into our monorepo in 2.8!
  • A fancy new image zoom feature in our front-end (thank you @416rehman!)
  • Many improvements and new features for our Dashboard, not least a complete re-working of the modules to use the Vite bundler (thank you Andrew!)
  • Microservice health-check route and reporting improvements
  • Lots of documentation updates
  • Improvements to our React Native app UI
  • Refactoring of a bunch of GitHub code in the front-end
  • Improvements to our autodeployment server for git SHA handling with builds and GitHub Tokens
  • Tons of dependency updates

Many hands make light work

After our last release I wrote that this group was really starting to come together as a team.  That teamwork was on full display during the 2.7.0 cycle, and I enjoyed seeing most people find their place in the work.

Hung, Thanh, and Diana worked at getting our React Native app ready to use, and Kevan is working on a PR as I write this post to automate deployment to Expo.  Amasia found some interesting edge cases in the Windows setup that we managed to document.  Hopefully I'll be able to install the app on my phone next week.

Jia and Roxanne have now bent Elasticsearch to their will, and our search tests are landed!  Their efforts revealed a number of bad bugs and issues that we managed to sort out along the way, and I know it also gave them more confidence to go deeper with our Elasticsearch integration.  I'd love to see them add autocomplete to search next (hint!).

Thanh and Francesco have been doing amazing things with our monorepo tooling, especially with pnpm packages, automatic deployment of sub-projects in CI/CD, and Turborepo integration.  I think the process and code is becoming clearer to them now, and the effects and benefits on our project are becoming more obvious.  I've learned a lot watching them work.  Using these tools well requires you to approach problems in new ways.

Duke's been as busy in our Telescope repo as he has upstream, helping to fix bugs with Supabase.  One great experience we had in this release cycle involved Renovatebot submitting a dependency update to Telescope for a fix Duke made to Supabase's postgres-meta package.  Learning how to support the entire software development ecosystem vs only fixing your own bugs is one of the most important lessons of this course, and it's great to have examples like Duke to point at when I need to explain it to future students.

To that end, I'm also really excited that Jerry got his dependency service working, which I saw run on production last night (very cool!).  It generates a complete dependency picture for us of all the npm modules we're using (and their dependencies) via pnpm's lock file.  We hope to integrate this with the GitHub API and use this as the basis for future students to find good issues and projects to work on as they learn their way around the open source world.

Tim's been laser focused on Docker and infrastructure fixes, and it's moved a bunch of things forward.  I tried asking Docker for support to let us host our Docker Images with them under their open source program, but didn't hear back.  So I'm pretty excited to start implementing our own Docker Registry next week with Chris Tyler so that we can go even further with improving our setup.

Joel's been working on security infrastructure to support the new Supabase services that Duke added in 2.6.  It's required us to rethink a lot of our current authentication/authorization code to fit with what Supabase wants. There's a lot left to do.  Quite a few students have been asking me to explain how all this code works, which is good because understanding the mechanics of SAML2, SSO, OAuth2, and friends is a great industry skill.  Also, I don't want to own this code on my own!

Alex has continued her work on community management, focusing on a Slackbot integration, continuing to plan and document our architecture, and experiment with streaming for Telescope.  She's even got people discussing the need for localization on Telescope, which is a topic dear to my heart.  Her work at community continues to inspire me to take the time to write posts like this, since community requires story telling and story tellers in order to help everyone understand who and where they are.

Tue has been digging through our backlog and solving some longstanding issues that have plagued our development.  First, he fixed Renovatebot, which had suddenly stopped sending us automatic updates (tldr: a branch got left unmerged, and blocked any new branches from being added).   He's also been working his way through reviews on our parser service (thank you Diana!), which is the last piece we need before shutting down our legacy backend in favour of our microservices.  It almost made it into 2.7, but I'm sure he'll have it done for 2.8.

We also ripped out our existing Users service and Firebase, replacing them both with Supabase.  It's always sad to see good code get erased, but it also speaks to what a great job Chris Pinkney did writing it in the first place: modular, well-designed code is easy to delete.

Speaking of previous contributors, we saw a lot of activity from a number of familiar faces. Josue helped a number of us debug multiple issues on staging and production, which was very much appreciated by me at 5:00 pm when our whole infrastructure fell over.  We also saw Cindy finally get her Docusaurus instance for Telescope.  She's been talking about doing this for years, and it's great to see it come to life.  I can't believe how many people got involved in working on it.  Clearly she's hit on something popular.

Our Sheriffs through this cycle did a great job.  Kevan and Joel both wrote about what it's like to be on call, helping the team and community get their stuff landed.   Thanks to Joel, Kevan, Francesco, and Tue.  It's not an easy job, but wow, it's critical.

I'll end by welcoming our first-time contributors, Peter Le, San, and 416rehman, as well as recognizing contributions from other contributors from the past: Jacky Tea and Abdul.  It's great to have so many former students coming back for more.  Apologies to anyone and anything that I'm missing in this post, it wasn't intentional, there's just so much to try and talk about!

We're going to be hitting our 3,000th Issue and Pull Request soon and it's great to see the project getting stronger as the months go by.  Maybe 2.8 is the release you'll get involved.