Recapping my journey at Endless- 9 mins
It seemed only yesterday when I joined Endless as a Papercuts Team member after being a GNOME contributor for a while. I was supposed to fix papercuts issues in the OS, package a few electron apps to flatpaks and overall work alongside the desktop team. I am happy to report that the experience almost 2 years now, has been a learning and a positive one that has made me a better developer and human by-and-large.
Endless is a mission. A very tough one, though. Closely working with the team, I learned more about the mission and goals; how our work is going to address the problem-space and K.P.Is to measure the impact made; it was sheer-motivating. In other words:
“Being a part of something bigger, a movement.”
Over the course, I got shifted to the core desktop team, where I started working directly in the desktop team. I joined the desktop team at a time where the OS had a huge delta(GNOME 3.26) with the upstream GNOME and we needed to fast-forward the rebase over three releases(GNOME 3.32). Naturally rebasing such a delta is no cakewalk, given the desktop team was dealing with fallouts from the rebases until the next 3-to-4 rolling releases. Retrospecting this particular time period, I think we learned three major lessons.
Rebase, Rebase, Rebase every six months
Upstream all the things!
Adopt upstream-first strategy where-ever possible
The first one is quite obvious as the rebasing over GNOME 3.32 was a very painful one. We took a note of it in the retrospectives and made sure we do not miss a rebase over an upstream release cycle. The second and third are complementary to each other. Endless has taken great strides in this direction. Projects that were intially started as a downstream project based on the various forms of requests from the field/ground reports, have been upstream-ed across various open source projects, be it Kernel, Flatpak, OSTree and many core GNOME components. Endless also proposes plans that might be useful to the community in general for e.g., I can recommend the talk given by Robert McQueen on Product Metrics & Respecting Privacy for GNOME from GUADEC-2019, where he talks about how GNOME can benefit on metrics front, using some of the pieces that has already been developed and tested by Endless in the field. This is important as the community gets some food for thought, design and refine their goals and probably helps to develop a roadmap for future releases.
Highlights of the work that I have been doing!
Disk space improvements:
Endless suffered from
ENOSPACE on low-cost systems where total disk-space is
32GB. This was due to the OS image size containing all the offline knoweldge-library content and having apps such as encyclopedias. A ton of work went into improving disk space across Flatpak, OSTree and GNOME Software.
Eliminating the use of double-disk space temporarily by flatpak on system pull - During a ref-pull for the system-repo, flatpak intially pulled into a temporary child repo and then verify/copy each object into the system repo. This caused the problem of transient double-disk-space usage. Hence an encyclopedia app of size 6GB requires 12 GB of free disk-space in order to get installed. To eliminate this problem, a special FUSE system
revokefs-fusewas introduced to mimick
revoke(path), in addition to a trustable-but-not-root user as the initial owner of the files being written to disk. When all objects were pulled in, the FUSE is unmounted (this makes sure that there are no open FDs to
pathhence no changes can be done to the underlying pulled-in objects), verification and permissions canoncalization (check for setuid bits) and finally hard-link it to the system’s repo (hardlinks are faster then copying) hence, overall improving system-pulls speeds while reducing ~50% I/O at the same time. A more detailed version of the approach is documented here and you can also checkout the flatpak-source.
min-free-space-size=X GBs” config parameter for OSTree repo - A gatekeeper check that makes sure that a pull doesn’t fill up the entire’s user disk-space in the background. Plays a important role when the user has opted-in for auto-updates of apps/runtime or the OS itself.
Low-disk space heuristics - Some disk-space heuristics and special behavior in GNOME Software to detect if the user will run out of disk-space if they proceed with an install or update operation.
Refreshing featured apps in GNOME Software without pushing an an OS upgrade:
I covered this in detail in my earlier post.
Password peeking support in GNOME Shell:
Endless ground reports always had a request for password-peeking functionality so that was already one of the top priority at Endless. Adopted upstream-first strategy hence, GNOME 3.36 is shipping password-peeking support. This was developed entirely on Endless’ work-time and was one of the top on list of things to watch out for next release at the last GNOME Advisory Board meeting at FOSDEM (as I have been told).
Collaborating on GNOME new lock screen:
Few pieces around the new user-avatar at login and lock-screen were contributed out of my Endless’ work-time under the guidance of GNOME Shell and Design team. Not only I started to enjoy developing the Shell(as it’s quite a challenge + fun), I started to take more initiative for shell-related work. Also, to get these specific updates, I strongly recommended following the GNOME Shell and Mutter development blog!
Hooking parental-controls across the desktop
Parental controls has been one of the important focus for Endless for a while now and I fondly remember working on it as it was one of the most productive work-period during the journey. The project is authored and being led by Philip Withnall to bring this to the wider community for availability.
Needless to say that all this were unconditionally accompanied by chasing endless bugs/crashes/builds or test failures across various system-components, resolving release blockers, rebases, packaging and dabbling with infrastructure to setup up the pipeline.
Lately I have been doing some basic metrics restructuring work that has been refreshing to work on and also to take a break from doing GNOME-y work.
Developing and enhancing soft skills
To be honest, 2 years back when I started, I took soft-skills for granted and I assumed I pretty-much have it - “How hard it could be, right?” I couldn’t have been more wrong. My myth were busted time and again over the course. It takes real work to communicate efficiently while constantly monitoring/maintaining that I am not directing my agreement/disagreement/comments as a personal attack towards a colleague no matter where in corporate heirarchy or team. I tried to demonstrate a kind and humble attitude towards everyone and the work that they did and I have pretty-much got the same in return. Looking back, I see myself how much growth I have internalized in this particular domain which I think will help me in maintaining relationships with the people I have worked with over time.
On remote work and managing stress
My position as Desktop software developer has been 100% remote. I worked from India (UTC +5:30) most of the times. My experience with remote work has been positive so far and my experience with couple of Google summer of Code rounds had made an eligibility-check of “being able to work remotely” (as Endless is my first-company to work for, after I graduated in 2017) while I was interviewing for the company.
Now to the section of managing stress and burn-out. It’s pretty much established that, at least at some point in work-life, one faces burnout and doesn’t know what to do about it. This was new for me as well. Working, inevitably brings stress especially when you are working against a release deadline and/or being in a deep rabbit-hole or just when things are not going as you would expect. Initially I used to go silent on things like this, fearing that it would affect my performance reviews and being looked down upon as an engineer; but to my suprise opening about it eventually, was one of the best things I have done. Not only was I provided with support but I was made to understand that this is normal and everyone suffers from it at some point in their careers. All we can do is learn from it, listen to warning-signals and take regular time-off to recover from it.
Before closing on this note, I want to just touch upon couple of more points about remote-work that I think are valuable lessons learnt:
Be clear in your communications even if it means over-communicating a bit.
Being remote is great and all, but also judge if the work isn’t quite async enough beforehand. If the work or iterations requires tight feedback loops to work it through and requires 2 or more persons in opposite timezones, I (personally) have found very difficult to work in that setup. Although this should not be a mandatory rule to follow but I think pro-longed periods of such kind of work might stress you out faster and should be avoided (or be made more async-ed), if possible.
On Endless next steps
As Endless moves ahead to restructure itself; being a full-fledged non-profit foundation, I wish nothing but huge success on their path forwards. Certainly considering working out all these years that have brought learnings and experiences about how the next billion of users will interact with computing; the decisions made in the resturcture makes total sense. I also want to give a huge shout out to the entire team at Endless who has helped my grow in so many aspects. Thank you so much.
Having said that, I will be looking for a job very soon. If you or your org. is looking for a generalist system-developer with track record in GNOME or associated technologies, I am eager to talk to you. Feel free to drop a comment down below or ping me at Twitter, LinkedIn or email at
Until the next post, Happy Hacking everyone!