From Telescope 2.5.0 to 2.6.1

by David Humphrey

I am not really sure how, but we are doing a lot for release 2.6! – Diana Belokon
So much has happened in week 3 for OSD700, and I had quite enjoyed it all – Roxanne Lee
We are now at version 2.5.4. Our fourth patch release in the span of one week! – Francesco Menghi

Today we shipped Telescope 2.6.0, followed a few minutes later by its younger sister release, 2.6.1.  I wanted to say something about what the team worked on, and how we did in the past two weeks.

Of Sheriffs and Team Collaboration

First, despite this being our second major release this term, I'd say that 2.6.x represents the first release where I've seen the majority of the current group really come together as a team.  During 2.5.x, people were still learning their way, leaning back to wait for others to do things, or pausing until someone told them what to do.  With the 2.6.x work, I saw a marked difference.

Part of this has been the new addition of weekly Sheriffs.  We've had 6 people take a turn at it so far, and I've been impressed with all of them.  The idea of having a sheriff is something I learned from Mozilla, and it's worked wonderfully in Telescope.  Our team this term is large enough to benefit from having a few point-people to help keep things on track.  I've watched as sheriffs have dealt with security issues, shipped updates to fix breakage on the master branch, worked with people to rebase and fix pull requests, and guide us through planning and triage meetings.  Students don't usually get to play a leadership role on software teams like this, and I think it's a great experience that people are keen to do.

Another thing I've noticed during the past two weeks are many more group programming sessions.  I counted 6 MS Teams video calls in the last two weeks, and I know there were a lot more on Slack and other platforms (some people are starting to think about experimenting with Twitch, too).  There are still some people who use DMs too much, but in general, I've been impressed to see the team getting into a more open, collaborative workflow.

Code Changes

In terms of what we worked on, you can see everything in the changeset, but here are some highlights:

  • Updates to our GitHub Pull Request Template
  • Automatic Docker image build and push CI workflow created
  • Integration of Supabase into our Docker setup
  • Addition of RSS-Bridge container to our Docker setup for Twitch feeds
  • Fixed routing in Traefik for microservices with static assets
  • Removal of cross-env and better environment handling in tests
  • Automatic linkification of feed content (e.g., YouTube descriptions), WITH TESTS :)
  • Integration of Octokit for GitHub API calls in our status microservice
  • Updates for many outdated dependencies, including MDX, node-fetch, and other broken module versions
  • pnpm workspace, script, Docker, and doc fixes
  • Lots of work on search tests, Elasticsearch mocking, and bug fixing related to what these found
  • Many Docker, docker-compose improvements
  • Many Dashboard and build log improvements
  • Improved dark/light mode syntax highlighting (hurray, finally!)
  • Fixes to our authentication and sign-up front-end code
  • Improvements to our GitHub front-end API React components and hook
  • Fixes to git SHA handling and reporting, as well as build log caching in our autodeployment server
  • Improvements to our log handling in production with Docker

There's more, but this gives a good sense of what landed.  In addition to what made it in, other pieces are in progress; among them:

  • a new docs site based on Docusaurus
  • a dependency graph service for discovering our dependencies, and later, opportunities to contribute to the ecosystem around us
  • a UI for our React Native app
  • search improvements in the back-end and front-end
  • SSO, auth, and JWT security improvements
  • completion of the parser service and microservice migration
  • turborepo monorepo tooling
  • Docker image, container, and Dockerfile improvements

And other things I'm forgetting.

Looking forward to 2.7 and beyond

As we work on individual issues, it's easy to lose sight of the forest for the trees.  Alex has been doing heroic work to begin organizing our discussions, issues, and pull requests into Projects.  This is the first time I've worked with the new GitHub Projects, and it's really powerful.  I'd like to see us get behind Alex with this effort, and try to map out our features.

I want to see the team move from working on individual issues in isolation, and start thinking about the opportunities that their work could unlock if connected to other efforts.  For example:

  • Diana, Alex, Cindy and others have begun to move us toward integrating media into Telescope (YouTube, Twitch).  We've got back-end support to get an Atom feed for Twitch users now, but no UI to enable it during sign-up.  That's an issue that needs to get filed and fixed: Users should be able to to include a Twitch stream or YouTube channel during sign-up.
  • Duc has moved mountains to get Supabase working locally and in GitPod.  It brings us so many new toys that we can slowly start adding to the back-end and front-ends we have.  For example, some issues we need to file and fix: our dashboard could use it to keep track of data in real-time (build info, service status, stats); our user and feed data could move out of Redis and into Postgres.  I'm sure there are a lot of ways we could use Supabase once we get more experience with how it all works.  Hopefully we can figure some of this out in our call tomorrow.
  • Jerry has been developing a microservice to help us discover and track our project dependencies.  I've long wanted to have this info so that I can point my open source students at open issues they can fix.  At the same time, Amasia, Roxanne, Joel, Jia Hua, and others have been working on our search back-end and front-end.  I'd love to see the search and dependency work get integrated, and connected to GitHub data.  Andrew has been doing a lot of GitHub API work in the status service that could help with this.
  • There's lots of talk from students about wanting a React Native mobile app, but so far I haven't seen much materialize.  I know people are keen to work on the tech, but it needs some leadership to get it moving forward.  Some issues need to get filed so that people can focus on concrete tasks.

These are just a few examples of what I'm talking about.  Lots of work, but we need to do a better job connecting it to the next steps and people involved in making an issue part of a feature.  I'm hoping to see more on this front in 2.7+.

Learning

I was talking with Tue last night on Slack about reviews.  He was asking about how to get better at doing them.  I've always liked Googles Code Review guide, and I'd read that.  But more practically, I'd advise people to look at review as a chance to learn how code works, and communicate changes to the team.  I've written about this in the past, and I think it's worth repeating.

During this release cycle, I reviewed a hundred or more PR requests.  Sometimes I knew the code really well, and other times I was seeing it for the first time (in a few cases, having to learn the technology as I read the code).

When I review something, I start by trying to get it running.  Does it work?  Next, I make sure that I can read and understand the code.  I ask questions: why have you done it this way? can you add a comment here?  what is the point of this change?  I approach code changes as something I'll have to understand well enough to maintain in the future (because I will!).

Doing reviews this week, I learned (or re-learned) lots of things:

  • Who can and can't push images to the GitHub Package Registry (hint: we can't)
  • How to limit GitHub Actions workflows based on paths
  • How Supabase does it's JWT and other keys for API access
  • Which services, URLs, and ports Supabase uses in Docker
  • How RSS-Bridge works and how to configure various feed providers
  • How to do URL redirects in Traefik in order to add trailing slashes to routes and how to include multiple middlewares on a router at the same time (aside: the Traefik docs need more examples).
  • How Supabase defines its postgres SQL schemas.  I'm somewhat intimidated by this, since I don't use postgres or SQL very often.  I'm going to have to learn a lot more.
  • How to mock Elasticsearch properly in node.js.  It's not hard, but it's also not obvious!
  • How to deal with specific package versions in pnpm, Renovate, and apk
  • How Renovatebot logs its scans of our dependencies
  • How Jest setup files work
  • How to debug our feed processor, and how 304s are handled in the back-end
  • Differences between building Docker images with and without buildx (hello M1 Mac!)
  • How Expo and React Native work (I've never worked with either before, but was able to get an app building for iOS, Android, and Web on my Mac, and I'm excited to learn more)
  • How next.js does CSS module imports, and how our theme loading works (I still don't fully understand it all)
  • How GitHub's webhook payloads differ from the documentation, and how to get a sha for both a push and release event

I've already forgotten the rest of what I learned.  The point is, I didn't write most of this code, but I still had to learn and understand it all in order to do useful reviews.  It's a great way to expose yourself to new technologies and techniques.  I hope more of the team will overcome their fear of reviews and not worry about doing it perfectly.  My reviews are never perfect.

I'm enjoying hacking on Telescope with people, and I know I'm not alone.  A number of our wider community members are also increasing their contributions.  It's always fun to work on something that has energy, community, and traction.  Telescope 3.0 has all of this.

If you'd like to contribute a fix, or get involved, let us know.  You'd be welcome.