It's been one year and three-ish months since my return to academia as a Ph.D. student at Michigan! From the time I moved out of Bengaluru, I've had this mix of pressure and expectation built up on how I would fare in this world that I know is going to be filled with huge ups and downs, a lot of excitement and pressure and most importantly, a lot of clashing of adapting to something new and moving on from the nostalgia of something old. I hope it gives you an insight into thoughts and expectations of what it means to be a first-year PhD student now :).

Courses

I was skeptical of how courses would go because I was not very disciplined during my undergrad and was afraid I'd repeat the same this time. Thankfully, I'm more mature now (only slightly) and understand myself better when scheduling things, knowing the gap between my ideal plan and what I end up doing (I'm not the best person in time allocation). There is not much to say other than this, but I have maintained a 4.0  because I wanted to prove that I can support a good GPA while meeting my other goals.

Research

What to work on: Unsurprisingly, I've been part of many discussions, brainstorming, and projects since my advisor and I (duh) were starting afresh. This is something I expected and was mentally prepared for. My advisor pointed me towards problems we could work on with a wide lens that fit the application in social good contexts while focusing on Human-Centered AI. Although I consider myself pretty flexible in terms of interests, I had a list of topics I wanted to work on, but as I dug deep, I slowly realized that other things require my efforts first, and after laying those foundations, I can move onto my next set of ideas. Back to the mindset of having a wide-scoped lens, I was ecstatic that we would explore with a high \epsilon value.

Publishing a paper: While I did not put too much pressure on myself, I certainly wanted to end my first year with one accepted publication to boost my self-confidence and feel capable of leading research problems individually. Thankfully, the stars did align, and CHAIR was accepted (as a workshop paper at CVPR and a full paper at IJCAI). I got to present this work at two poster sessions and give a talk at IJCAI, which felt really good because I've always thought that I've lost the touch of giving talks and generally speaking to more than three people (I used to do a lot of public speaking while I was in middle and high school and have barely done anything close ever since).

Candidacy: CSE UMich rules dictate that PhD students must present their research in an hour and a half session before the end of their second year to be candidates (this includes finishing courses). Luckily, I did pass it :). However, the effort behind this was giving many practice talks to many people and a couple of plushies on my sofa as my audience every night before the talk. More on this later...

Conferences and Travel

Luckily, I've been to three conferences during this time - NeurIPS, CVPR, and IJCAI, with posters at all three and a talk at the last one in Jeju, South Korea. The attendance of NeurIPS and CVPR was 12,000! One of the best moments in both these conferences was meeting so many people whose work I've admired and respected. Contrary to a lot of networking advice for PhD students, which focuses on productive meetings at conferences, meeting the greats means a lot, especially coming from a country where accessibility to outstanding scientists only happens at top institutions and with a lot of reluctance. I've made some good friends at these conferences, built the confidence to approach strangers, and, most importantly, really complimented people whose work I've been reading.

I look forward to meeting more cool people and making good friends at these conferences; it is somewhat concerning to me that I have to do a lot of homework and preparation so that these conferences are productive, or else I will waste a lot of time and effort trying to find relevant work.

Skills

Talks: I've realized I don't have the natural confidence or ability to give a good talk (which I imagine is the case with most people). But good news—I realized that soon. I've been told that the PhD time is when these things can be worked upon :), so I've been trying to do so for the past few months. Before all my talks, I made sure to practice my talks at least half a dozen times so that I would be fully aware of my slides. I have come up with metrics that measure how confident I am about the talk. Movement around the talk space * ( 1/Number of times I've looked behind me to see the slide) is an excellent metric to maximize my confidence in giving that talk. I also scored the highest in my Advanced Data Mining class for presenting the assigned paper. It's not much, but that's a win for me. I've also been told that my Computing in Society talk aimed at freshmen and sophomores was well received, so it was not bad all in all.

Writing: I haven't entirely been happy with this. I vowed to be regular with my blogs at the start of my PhD, but that hasn't been the case. I'm hoping blogging can be regular starting next semester because I will hopefully finish my courses.

Confidence: I don't think I was completely confident about handling all the facets of being able to do a PhD, but I'm starting to turn that narrative around. I've started to compare myself with others less than before (easier said than done when every signal from everyone around - peers, internship, etc. is based on comparisons).

Hacking: Placed 2nd in the Google x MHacks with so many caffeinated undergrads. I'm now a twice winner - including one while I was at MSR. I was happy with the idea and execution - I learned how to build a VSCode extension!

Personal Wins

I'm glad I have traveled more than I did in the past two years in India. I've visited three continents (and countries) and eight cities. All the travel has made me an efficient packer and better at planning travel, which is very handy when attending conferences when the schedule is so tight. I also played guitar for a hundred people for the first time, which has been on my bucket list for so long, and I think I did well. (Side note: I highly recommend every PhD student to have an instrument with which you can unwind)

What am I looking forward to

More cool research, obviously (read in Snape's voice). But hopefully, impactful research and focus on improving my communication skills (scientific and general purpose). While I want to say I'd like to reduce comparing myself to others, it's a hard promise to keep, so I'll try. If you know how to do that effectively, please email me. I'm planning to write these reflections every six months or so, so I'll keep you updated on whether they've reduced (I promise I'll be honest)

Chanakya was finally accepted at NeurIPS 2023! This meant preparation for the poster, the video recording, and most importantly, travel to my first big conference! Attending the conference was exciting and life-changing in unexpected ways.

NeurIPS (Neural Information Processing Systems), is one of the top conferences in AI and among the top 3 in the world based on Google Scholar's report. It was held in New Orleans, Louisiana, USA this year for 6 days from 10-16th December. I was at the conference for three days. The agenda was simple - a) attend my advisor's tutorial presentation on the 11th, b) co-present our poster on the 13th and c) meet everyone whose work I've long admired and respected to get some of their opinions and perspectives on the problems they're currently working on. Well, I think I did all of that successfully!

The experience of NeurIPS was very overwhelming! The people, booths, and events all made it a worthwhile trip.

Venue: Nothing like I ever expected

The convention center is located right at the bank of the canal. I was lucky enough to grab lunch with some amazing people who were at the tutorial with this as our view.

I never expected to see that many number of people attending the conference. The first indication of the number of people was the fellow passengers on my flight from Fort Worth to New Orleans. The flight was filled with people going to NeurIPS and all were easily identifiable by the poster tubes they were carrying. The bus from the airport to downtown New Orleans was filled with people carrying posters. The true spectacle was the number of people at the session where the awards were being presented and it looked like at least ten thousand people were attending it. I kept referring to the annual exhibition that takes place in Hyderabad called Numaish to people who knew about Hyderabad to give a visual guide of what it meant to attend the conference.

Socializing

Whova was the "official app" that you could use to socialize with people attending the conference. I was pleasantly surprised by how many people were active on it and I met quite a few people through it. It is always exciting to meet so many people with different backgrounds which makes for interesting conversations. There were also some parties/dinners hosted by companies like Weights and Biases, Microsoft, Google, etc. (although the events hosted by the big companies were exclusive). I had a lot of fun attending some of these and met peers who turned out to have a lot of common things with me than I expected (if you don't already sense it, socializing is not my forte - I've been very shy and this is something I'm trying to improve upon)

One major highlight - I met Yann LeCun and Chris Bishop at the Meta and Microsoft booths respectively! Although it was quite brief, I thoroughly enjoyed my interaction with them.

A major takeaway of mine was to plan every day and schedule a calendar event with people you want to meet with. I did the same for the last two days of my conference it was great. It helps that most researchers are really helpful and they do tweet out that they're interested in meeting students. I reached out to everyone whose work I've been reading and caught up with them at the conference. If you're reading this and happen to be one of them, thank you so much for your time!

Poster Presentation

One word: amazing! We had so many people visit our poster and it was not as tiring as I estimated (maybe it will be once I'm through with a couple of them, but I hope it doesn't fade my enthusiasm). It was a 2-hour poster session from 5-7 PM. Luckily we were able to wrap our presentation by 7, but I did see many people still presenting their posters at 7:30 PM, something to be appreciated in my opinion! The most asked question that I did not anticipate was "What is Chanakya and why was the paper named after that?"

Swag Haul

I will admit unashamedly that I dedicated two hours to grab some of the best swag offered. My favorite ones were the beanie from IBM, a wireless charger from Determined.AI, and a mug from Intel AI with a printed photo generated by some diffusion model they were demoing on their hardware which you were allowed to customize (it was a cute samoyed for me if you were wondering). The hydro flask from DE Shaw (the timing was great, I was in the market for a flask :) ) We were given quite a lot of T-shirts too, and I was left with no space by the end of this trip.

All in all, I thoroughly enjoyed this! Academically, I'm not sure how useful it is to conduct poster sessions that run this long and there are so many good papers that there just isn't enough time to attend them all. That said, I'd highly recommend in-person NeurIPS if you ever get a chance.  

December has been particularly challenging, especially due to the long application submission season, and now, as I am recovering from a week-long sickness. Regardless, as all dark clouds have a silver lining, I've been provided the time to reflect on my journey, relationships, and personal self over the past few years, 2022 more specifically. This year, as usual, has been a bumpy ride - from moving to Bengaluru to living on my own(again, post covid) and pushing myself to be a more social, receptive human being, I've realized there has been significant professional growth and a considerable amount of personal growth this year, all credit due to the amazing camaraderie at MSR. Although this is good, I think there is a lot of room for me to grow and push my comfort zone. Professionally, I've been exposed to a dynamic work environment ever since offices opened up earlier this year - as a researcher, a team member, and handling responsibilities that have highlighted my lack of people skills(which is where I suffer personally, too). I'm looking forward to 2023 with improving my environment as the focus.

MSR and Research

via GIPHY

To this day, whenever I glance at the Microsoft Research logo at the entrance, I am momentarily left wondering how I ever got here, which either occasionally proceeds to either "past Vaibhav would be proud" or "me, here, really?". Either way, these thoughts end with a moment of gratitude - I am grateful for the opportunity to be a Research Fellow, to my parents who've been supportive throughout the journey, and a special mention: to all the new friends I've made at MSR - the conversations, knowledge, and support that I learn from them is something I'll always cherish into the next phase, whatever that will be. But then,  the never-ending stack of tasks shines so brightly that I'm quickly put out of the trance, and reality resumes. Research has been fun - it's been a mix of understanding the research process, diving into new fields, reading up on areas I normally never would've explored if I'd decided early on what my graduate studies interests were, and most importantly, good engineering. Interacting with domain experts has been the key highlight of all my experiences, substantially improving my ability to adapt depending on the audience. TL;DR no complaints with research; that aspect has been good.

All this said I realized my current work ethic and workflows are not sufficient. I always thought my semi-structured workflow was sufficient and ideal because my structured workflows always suffered the same flaw - they were all too rigid, and I ended up scraping/re-designing them most of the time. But I failed to realize that I'd let loose of the structure a little too much that it allowed laziness to permeate slowly, causing some miscommunication, some missing deadlines, etc., that would not have happened otherwise. So over the past few weeks, I've been obsessing over these and trying to understand how I could've done it better. By a stroke of luck, I had come across this tweet during this period and decided to take a chance on it, given I had nothing on my reading list then.

I have to say, this book, "How to Take Smart Notes," talks about some of the problems I'd been having in organizing my thoughts and how not having a structured writing regimen decreased my research productivity. I review it in depth in my other post, but briefly, the author motivates how the traditional advice recommends a linear process of writing, which doesn't apply to research and writing in research(articles, papers, etc.). The author further introduces the "Slip box" method or "Zettelkasten" method, as the productivity internet now has embraced the original german name. While this is slightly archaic and needs a little modernization, I've decided to use the principles and the core framework behind this and apply it to managing my life in general.

Improving Communication - pictorial and verbal

I've also decided to understand and improve aspects of communication - personal and professional. Academic communication doesn't just involve speaking to your collaborators, but also things like the ability to communicate your ideas at talks, and in papers. In my case, scientific communication via images and diagrams has never been my forte - the constant self-loathing of not being able to communicate some ideas via pictures in presentations and papers has been very frustrating. I've never been into any art that deals with drawing or painting - partly because I find it hard to express anything via pictures and mostly because I'm horrible at it. Growing up in India, the benefits have never been motivated sufficiently to try art, and for someone who is abysmal at it, I never really cared about any of it until it did, which is now. Simple, lucid diagrams can significantly impact the quality of all your talks and presentations when communicating the core idea, especially when there is a time constraint and the audience can vary dramatically in their background. (Update from the future: I received my first good diagram compliment, taking that as a win)

Another aspect of communication is verbal, delayed communication with people in different time zones and regions. Most of your collaborators are reachable(in person), but not everyone - which makes the hybrid style of working extremely difficult. It was never tough when everyone was remote, scheduling was simple, communicating was easy, and calibrating to everyone's schedule was minimal because all you had to worry about was checking calendars. When in the lab, the calibration must be taken to a whole new level to comply with everyone(people have errands in the middle of the day, hard to accommodate beyond your preference, etc.) - there is so much out of your hands that the best thing that can be done is assume the worst case scenario and run with it. While I experienced a lot of good whiteboard magic with my teammates(something about the physical discussion that makes it engaging), improving my ability to handle hybrid communication is also something I need to improve upon (update from my future self - I've been noting down meeting notes, exact communication schedules to make sure there's never a doubt for all the parties involved, so that's good).

I believe working on building a strong sense of discipline is something I'm looking forward to in 2023. Heavily inspired by Dr. Vinitsky's blog on Research Productivity and Ph.D. expectation setting, I want to be in that develop a structured sense of discipline(in work, collaborations, and personal goals) throughout the remainder of my tenure as a Research Fellow and, more importantly, into my Ph.D. studies.

Personal Goals

Increasing Writing

My writing has not been up to the mark as I'd previously hoped. I currently have two blogs, one where I talk and rant about non-technical stuff(this one) and Research Recap is my academic/research blog (it was named VLog earlier, short for Vaibhav's academic bLog - terrible name), where I try to summarize essential takeaways from papers I find interesting(and what their potential implications are to my focus areas). Hopefully, over the next few months, I'll try to write bimonthly on the personal one and bi-weekly on the technical one. I have plenty of posts sitting on draft in my technical blog, accumulating review debt. I plan to revamp the blog's look, publish the draft posts, and gain some writing momentum soon.

Update from the future - started writing on Research Recap, covering interesting RL and optimization-related research. I've also changed the look of it, thanks to Quarto. Hope you find it useful.

Running

I've never been able to run. Even at the gym, I don't spend more than 10 minutes on the treadmill. Furthermore, I prefer weights - which makes hopping on the treadmill less appealing. As a fat kid, running has always been tough - mentally and emotionally.  I've decided to take Sameer's advice (someone I deeply respect and consider a mentor) - I'm going to actively try to improve my running and running form to improve my stamina and hopefully go to a stage where prolonged running gives a "high" and not stress.

Uncertainty of 2023

As a potential Ph.D. program applicant, it is incredibly nerve-wracking waiting for the results, with every waking hour being soaked in constant uncertainty. While a week in bed recovering from the fever has helped me overcome some of the anxiety, I'm slowly getting accostomed to it. I've now come to accept it - no matter what the outcome is, it will be fine - I'll try again for the next application cycle in case I don't get in anywhere and, meanwhile, explore more things. As I write this, there is still fear and doubt within me, which I guess is natural, and I hope it does drive me to do better, regardless of the outcome. Will I be devastated if the worst happens - YES, Absolutely YES! but I've accepted that I will recover eventually. There, I'm finally an adult now.

via GIPHY

Work at MSR is getting more interesting - from reading optimization-related research to learning about how AI is being used to tackle Climate Change when I'm not coding, I was feeling slightly burnt out. But then, we had a vacation! Microsoft Research graciously accommodated and arranged for all the Research Fellows'(RFs) here at Microsoft Research India to take a trip to Shillong! This was very timely since I barely getting time to hangout personally with the incredibly talented RFs and RSDEs, so this trip is where I was looking forward to knowing more about everyone I've met in the last couple of months since office has opened up and have fun with them!

I also wanted to give a shot at photography because I've been looking to learn more about it and wanted to understand what my instincts looked like. Here are a couple of photographs and videos I've shot on my iPhone XR (Spoiler: I'm a complete newbie, open to feedback and lessons).

We first visited the Umiam Lake straight out of the airport after a decent lunch in between Guwahati and Shillong. The view was breathtaking

Day 2 is when we visited the cleanest village in Asia: Mawlynnong and the adjacent Single Root Bridge. We also pit stopped at a bunch of viewpoints, one of them is captured right below.

Day 3 was Caves and Waterfalls' day. We visited the Elephanta caves, just caught a glimpse of the Seven Sisters' Falls but couldn't go anywhere close by because of the low visibility

The current state of Jetson devices through NVidia's JetPack is based on Python 3.6 ( including all the pre-built wheels and docker containers ). Python 3.6 is increasingly breaking a lot of dependencies solely due to the fast pace evolution of Python every version. This post serves as a guide to anyone setting up their Jetson for ML experiments. The primary motivation is it took me about 20+ hours ( yes, I didn't get a good night/morning sleep ) to setup the device without libraries breaking. To prevent that from happening to you or anyone in the future, here's a small guide:

Hardware Setup

1. Ensure a stable power source
2. SD Card or external SSD a minimum 128 GB (recommended: anything above 256GB)
3.  Wired mouse + keyboard + monitor for the initial setup
4. Flash the Sd card / SSD with the image from the official website and set it up as per their instructions (not covered here).

Installation(important ones)

The estimated time for this section takes around 6 hours to complete.

1. Install Python 3.8+ - install your required/favourite version of Python(as of today, I'd recommend 3.8) - sudo apt-get install python3.8-dev [Estimated time: -5 mins]
2. Create a virtual environment from this newly installed python version: python3.8 -m venv venv (the second venv being the name of your virtual environment) Note: If venv is not installed, python3.8 -m pip install venv [Estimated time: 1-5 mins]
3. Install cmake, ninja, and some other important tools and libraries. sudo apt-get install cmake ninja libopenblas-dev libatlas-base-dev gfortran [Estimated time: 2-10 mins]
4. Install the required python libraries: pip install numpy pybind11 scipy scikit-learn scikit-image [Estimated time: 1-5 hours - Scipy is built from source]

Installation(optional)

Estimated time for this section takes around 7 hours to complete.

PyTorch: Building from source [Estimated time: 5 hours]

TorchVision: Building from source [Estimated time: 30 mins - 1 hour]

git clone https://github.com/pytorch/vision && cd vision && pip install -e .

MMCV [ Estimated time: 30 mins]

Follow the instructions on their website

MMDet [Estimated time: 10 - 30 mins]

Follow the instructions on their website

Make sure you've tested your required methods through and through(a successful import doesn't necessarily mean a successful installation). Now, to the essential part: Memory management. My tasks involved heavy memory utilization, multiprocessing, etc. all of which requires some more intervention.

Memory

Shared memory

Increase shared memory capacity: sudo mount -o remount,size=8G /dev/shm  This allows slightly larger amount of data to be shared between processes when doing multiprocessing related tasks. Remember, this must be executed once every boot.

Swap memory

Disclaimer: the boot sequence initiates a swap memory test on reboot which always fails on my Jetson, but the boot from power off works fine. Proceed with caution

1. sudo fallocate -l 8.0G /swapfile
2. sudo chmod 600 /swapfile
3. sudo mkswap /swapfile
4. sudo swapon /swapfile
5. sudo nano /etc/fstab
6. sudo <your_fav_editor> /etc/fstab and add /swapfile none swap 0 0 at the end
7. Shutdown your Jetson and boot it up.

This enables handling large datasets and models with ease when compared to the default configuration. An additional caution when using the dGPU, the memory is shared with the CPU, so the memory optimizations there must purely be code-based(sharding, lazy loading, etc.)

General tips

1. Use Vim/Emacs. Avoid VSCode as much as possible
2. Use SSH. Avoid using a monitor as much as possible
3. Docker comes pre-installed, but use it only when the load is not a lot.
4. Keep backing up files/results/etc.

Recently, I've shifted to a complete Windows/WSL dev environment - both in my professional and personal work, with the Zephyrus G14 replacing my MacBook Air 2015(8G+128G i5 5th gen). Hands down, the MacBook Air was and is one of the best, most reliable devices I've ever worked on! The entire MacOS environment is the best dev experience anyone can ask for, but it's time I said goodbye to a wonderful machine. I was and still am a huge proponent of MacBooks for a lot of dev work, hence this transition was tough for me. This decision involved a couple of factors which swayed me towards a Windows laptop:

1. I needed a new one(obviously) - the Mac was having boot issues/stability issues, etc. It's time I shifted from an almost 4.5 year old laptop.
2. I needed a machine with some decent NVidia GPU(leaning towards one from the Ampere series) to test GPU code and play some occasional light weight games when I'm really really bored.
3. From my experience working on the Thinkpad T450 for work, Windows + WSL covers everything that's required. It was time I reconsidered why Windows could be a better environment.

While I needed the laptop to be light weight, the decision to get the G14 was pretty obvious and one of the only few laptops that actually fit the requirement.

Review: It's pretty good

Specs: Ryzen 9 + 3050Ti, 16GB RAM. I'll be very brief about this:

1. The light and small form factor feels really good and fits perfectly for my requirements.
2. Windows + WSL had no hiccups and the Armoury crate gives good control over various hardware stuff.
3. Typing experience - really good(good key travel, comfortable key presses - Note: I'm light on touch) - Two subtle things which I liked and didn't like: 1) The W key has a ridge similar to the F and J keys (not sure if it's a common thing among gaming laptops) which I really liked; 2) The control and function key placement on the left side of the laptop - I think Thinkpad does this right with the Fn key towards the left and Ctrl key beside the Windows key. Call it MacBook bias, I think the natural tendency to access these keys tis through the thumb and not the little finger. Not a deal breaker, but just something I didn't like.
4. Good trackpad, fingerprint sensor (people have said otherwise, I think it's good)
5. Great display, awesome anime matrix behind!
6. No webcam - which I prefer but also the room for webcam on top of the screen is not that much, hence a bad webcam clamp can ruin the display.

I'll be updating this post with images soon once I get some time(hopefully with some cool Anime matrix stuff!). Overall, really happy about this decision.

Desktop Wallpaper: Photo by Mado El Khouly on Unsplash\

Lockscreen wallpaper: Photo by Anders Jildén on Unsplash

Earlier last week, the Microsoft Global Hackathon 2021 kicked off with a lot of folks at MSR India being excited about it! I've been hearing how previous hackathon winners in various tracks have managed to turn it into an actual, actionable project  which have since being continued or even completed! This sounded both exciting and a lot of expectations to live up to! But, in the end for me I entered it with the intention of learning various (hopefully non-technical) perspectives that goes in building solutions for people (spoiler alert - it was more than I expected and learned beyond what I imagined).

To begin with, I joined the Democratization of Sports and AI led by Sameer for one very simple reason - it looked fun! "Standard research" in neural networks for computer vision doesn't often involve datasets beyond the standard benchmarks. Watching them run in real-time provides an intuitive sense to the "vision" aspects and it's possible points of failure. For example, if I were to mention that a neural networks' runtime latency is 50ms, 50 ms looks like such a small number. Now, on a 2 minute video(i.e. 120s) containing 120 * 30 FPS = 3600 frames/images - running this model on this on an edge device would be 1800 seconds or 30 minutes! Based on the context, all sorts of tricks come into the play (I'd call this engineering finesse - tricks which research doesn't encounter/seem worthy a problem but can make or break your pipeline).

Our hackathon schedule everyday went something like this: Wake up -> Morning sync -> Work/Research on the hackathon code -> Do your day to day stuff -> Nightly sync. We had 2 syncs everyday to cater for people from different timezones. While the developing team was mostly from MSR India, this year there were roles for people who are not developers but can help in various other important aspects i.e. ideating, designing, advising, etc., all officially grouped as advisors. Contrary to my previous hackathons - the team we had was a lot diverse in term of skillsets. While I'm not very used to explaining computing concepts to non-programmers, this was definitely the easy-wording perspective always helps, regardless of the kind of audience you have.

Leading the team was definitely a tough experience - I'm not fully aware of each of my teammates' expertise, their comfort level with the task at hand, their communication preference, etc. which makes it easier to understand and co-ordinate in-person.  But, we finally did it and had a decent working prototype. I'm currently en-route making it a demoable Streamlit app for everyone to use. Here' a sneak peak:

Having said that, working and leading a technical team fully understanding needs from running coaches/experts and alongside our star designer Manjusha was a growing experience for me. This might have been more fun in-person, but hey, this is the best we can do now and I am eternally grateful for Akshay for encouraging me to join this team. Hopefully, I'll add a public demo link, but meanwhile, hope you had a good read.

Internet is a small reflection into multiple personalities that could exist when humans are exposed to different environments. That being said, a recent thought I had initiated a train of thought: what if we restarted the way internet is accessed and everyone is given a social security - an identification which acts as a gateway to use any/all services and how would that work ? There are multiple assumptions that shape the scenarios when relaxed, dramatically change the internet as we see it.

Assuming we can build a fresh, new internet and assign every person on earth a unique number - their own key to the internet and all of it's infinite knowledge, how would the internet be structured ? This encompasses the democracy of the internet, it's safeguards, it's limits and most importantly, it's governance and sustainability over a long period of time. As far as my knowledge limitations go, I broadly see three huge problems - 1) Control, 2) Laws and 3) Infrastructure.

1) Who controls the new internet?

Well, this is by far the only question that matters. All other issues are only a consequence of mishaps/invasions that can happen are an aftermath of mis-governance - similar to the answer to the question: would you chose power, money or fame? Power gets you everything! Back to the do-over scenario, there's no exact answer. While there could be a case for democracy, at the end of the day democracy is only as good as the worst apple in the society. This is clear in most modern societies as a lot of countries around the world still face a lot of issues while being democratic, with power being abused by wielding internet as the ruler's whip. A UN for the internet is a good place to begin with maybe, since there are fundamental laws for the internet which every country must adhere to , similar to fundamental human rights everyone is entitled to regardless of the nationality. One could argue that this leads to leaders of countries having a final say over internet access which excludes all of their citizens, with the people having no say in the matter. Even in blockchains, the 51 percent attack is an example where democracy can be misused.

From a different perspective, a fixed set of people sounds appealing, but with proper verification and regulation protocols with ethical oversight, this could be a good solution.  

2) Laws - Definition, Jurisdiction and Decision

This brings us to the next topic: Laws. While some fundamental laws can be established for Internet, internet crosses borders. The judicial system varies state to state, let alone country to country. All countries(and states if it's a country-state government system) must agree to be bound by an international rule of law and it's outlets of justice, as opposed to their own. This is an invitation to people yelling about lack of a power grab and control over the outcomes, etc.

3) Infrastructure

Infrastructure has been a long standing bottleneck for small companies to compete with the big tech giants. Let's take Google Drive for example - Google houses some of the biggest data centers in the world - from large scale storage to compute, they have it all. Imagine being a new player in cloud storage, only to compete with the likes of Google, yikes! But, money is the solution to that, which ironically is the solution to a lot of things according to people these days. But when I say infrastructure, I mean software infrastructure. If cloud storage is meant to be turned into a business, open source protocols must ONLY be used in this new internet world. That means the only control you have is how users interact with your service and the scale of your infrastructure. The core software protocols that dictate the privacy, security must be standard. On the flipside, this makes business that much more demanding, because the fighting ground is now slightly levelled.

While storage and storage privacy is only a small section of the internet, there larger take is the push towards open set of protocols - security, compliance, etc. that is up for scrutiny and therefore, improvement for the greater good.

After all these, is it still feasible?

Probably not, because fundamentally, the internet community is susceptible to some of the flaws we have as a society in this modern world - discrimination, money/power imbalance, lack of co-ordination/co-operation living with foreign communities. At a base level, it's these issues that actually matter.

OpenAI has recently taken over the tech circles, surprising everyone with their code generation models. A few weeks ago, I received access to their Co-Pilot and I'm surprised how much boilerplate it can predict. Due to the non-triviality of my day-to-day code, having Co-Pilot write my suggestions doesn't seem to help me much. On the flip side, Co-Pilot shines in assisting me to write code for research. It can recite torchvision ResNet from a one-word prompt, oftentimes even without any mention of ResNet(I imagine this being the case because ResNet could be the most recurring code on GitHub after import torch ). So, it can be safe to assume that after acquiring a lot of users and a lot of engineering efforts, the Codex can be tuned to do some trivial tasks almost perfectly. So how does this translate to apps that are powered by this technology to an average user? How do the UI/UX design principles have to be established?

This post is partly inspired by the above Codex demo. As you can see, Codex generates code that is executed at runtime based on the user's natural language input. While this in itself is impressive, imagine all the apps powered by a GPT-* model having some sort of natural language being turned into code that does something on the app to help you out - this introduces new challenges in terms of UX. In apps like these, streaming data is a first-class citizen to any direct changes on the app and unlike current apps, the data returned might not always be perfect. For example, in an instance where for some reason, if red was inferred as bread, there's an additional confirmation and correction delay introduced which makes the UX worse than just doing the task yourself. This introduces a paradigm where prediction latency and user expectations must be synergetic and can be elegantly handled via strong design principles.  This paradigm is slightly different from saying Siri/Google assistant where the actions performed are essentially a smart way of executing code humans have written.  I predict this is a start in a transition to apps that stream natural language input to have the desired output with 5G's adoption growing now more than ever, complementing these scenarios.

So, how does this fit into the general Human-Computer Interaction(HCI)? This brings me to another class of models, Perceiver IO by DeepMind. Perceiver IO introduces techniques to work with multi-modal inputs - language, images, audio, etc. all leveraged by a single network given the right task and dataset at hand. Imagine now a model having the ability produce generate outputs based on different sources of input and all of them synergetically having the ability to give a user what they want! What if the next Codex built on GPT-x looks at your face and based on the natural language prompt, outputs code depending on your mood, etc.(personal note: I did get irritated when Co-Pilot just gives random output while I'm thinking) While this is an absurd example, the possibilities extend to how a robot perceives different inputs - human, environmental, etc. all with one model are endless!! A large-scale GPT with non-blocking multi-modal abilities can certainly have a much broader impact in terms of human-robot interaction as opposed to the current limited human-edge device interaction.  Non-blocking in this context would be the model would still be able to perform some things while missing data from one of the modalities it supports. HCI would here include safety, ethics, etc. all as primary focus points rather than just something in the background.

These models might just bring the revolution in design principles around HCI and UI/UX with future apps and developer tools that directly work with natural language input ease the entire experience with possibly more than just language!

Simply stated, Goodhart’s Law states that “When a measure becomes a target, it ceases to be a good measure”. History has a lot of examples where this is as clear as daylight, but people fail to realize that they’re probably experiencing it in their day-to-day lives and have never realized it. A good example of this is the popular Cobra Effect which sounds very obvious in retrospect and I think most people listening to this probably wonder why they obviously didn’t see it coming. History has a very funny way to present itself in society - people study to learn from previous mishaps, but can only predict repetition and correlation in hindsight rather than incidents happening right in front.

Talking about deriving parallel’s from history, the current society has a lot of similarities with Animal Farm by George Orwell. If you haven’t ever had the opportunity to read it, I highly recommend it. This book parallels how a couple of farm animals rebel against the humans for their inhumane living conditions ironically takes a full circle like the Russian Revolution. This history lesson while being very well known, there a lot of interesting parallels occurring in the current society that mirrors the incidents in Animal Farm. While I don’t want to be political and deviate from the central theme, one of the takeaways from this story is that history only reminds us that we’re all susceptible to Goodhart’s law, no matter how much society changes. From standardized tests to the chase to achieve perfectionism, we are doomed to fail spectacularly if this continues to grow. This does beg the question - can we ever escape it or are we destined to always loop in this vicious cycle of optimizing and destabilizing our lives?

Frankly, I’d like to be optimistic, but modern society wields power to the wealthy and penalizes the poor. There exists a large divide in ideologies between the people in control and people being controlled and this will only increase, probably causing one of the biggest societal disasters where everyone would’ve seen it coming and yet, no one acted on it. From extreme right-wing behavior to the oppression of those who are affected most by the very societal rules that are supposed to protect them, our attitude as members of society towards this pretty concerning.

Goodhart’s law is probably the curse of objective benchmarking - the objectivity is lost once it’s considered as a benchmark, kind of ironic and self fulfilling, don’t you think ?

It appears in many forms: it’s termed as overfitting in Machine Learning, probably something else in a different field in a different form, but I’m quite sure it’s a widespread problem. Circling back, is it an inevitable flaw or a reminder that there can always be change if we collectively realize our mistakes?

The Scout Mindset by Julia Galef introduces the concept of ideal rationality traits a human must possess and terms it The Scout Mindset. The book first introduces the two paradigms of thought processes - The Soldier Mindset and The Scout Mindset based on how one uses their reasoning. The author Julia further goes on to lay the groundwork - on why everyone behaves, reasons, and thinks the way they do, which is often complicated than a binary decision and ends with how to begin to think like a Scout. There’s a small bit of irony that when you do read this book, it makes a whole lot of sense when seen from an open mind rather than sticking to your beliefs of rationality.

How I found this book

Incidentally, I’ve come across Julia first rather than the book itself. Julia has a (severely underrated and lesser-known) YouTube channel where she discusses philosophical topics which in my opinion a fair bit of the current population must think about. While she has talked about the Scout mindset earlier, this book collates everything and a lot newer concepts in one book.

Who should read it

Frankly, anyone who’s interested in philosophy revolving around humans, decision making, and reasoning is the perfect target audience, but I’d especially recommend it to age groups on the cusp of adulthood where opinions and beliefs have consequences.

Why I liked it

A lesser-known fact about Julia is she’s worked with OpenAI, one of the few AI companies I know of who deeply(pun intended) think and invest in the policies surrounding AI, which in my opinion makes concepts in this book is a good-to-know when approaching this area. She first introduces the paradigms of reasoning and establishes what’s ideal rationality which I conclude - is not the be a scout all the time, but rather just incrementally move towards being a scout and occasionally take a step back when faced with conflict in beliefs, rather than doubling down on it. Each concept has a lot of examples from history, which I love the most in this book, apart from the occasional bayesian and programming references she makes which just appeals a lot more to me.

I highly recommend this book. Feel free to drop me a mail or leave a comment below on your thoughts about this book.

Let’s rewind time a bit. 10 years ago to be exact. I was a shy, 11-year-old nerd who loved going to school. Yep, I was that guy. My typical day in the life would be:

1. [6:30] Wake up in the morning for school,
2. [9:00] Attend classes and solve the Rubik’s cube occasionally if I’m particularly bored,
3. [15:00] Complete any/all homework early on so that I wouldn’t have to worry about it later
4. [17:30] Watch TV, study or sometimes - play(I didn’t have a lot of friends since I wasn’t comfortable around people and I wasn’t sure if anyone liked me either)
5. [22:00] Sleep

At this point, I’m probably describing the approximate schedule of a lot of Indian students. I’m not sure if different countries face this, but a lot of Indian students are now asked - “What do you want to do when you grow up?” This is a serious question and people asking this aren’t just throwing such questions around because they’re trying to get to know you or make a casual conversation, they’re waiting for your answer so that they can predict and map your path for the next two decades - including the subjects you’ll study, universities you’ll study at, the salary you might earn, etc. Now, growing up with no siblings or people slightly older than me that I can talk to and look up to, who have been through this mental state at some point, I had no idea about any of this - quite literally!! I did not know what it means to work or choosing a major. Having perspective or at least, the concept of knowing what it means to have perspective on topics like this is not often talked about or looked for. The working assumption with the current Indian society is that kids have perspective, are mature and informed enough. Do keep in mind, it’s not a question of being forced into mainstream Engineering or Medicine study, it’s more to do with the circumstances that come with this i.e. lack of self-awareness, perspective, and more importantly, lack of any maturity. While this issue looks like the parents or teachers who most influence our lives must be blamed, it’s more systemic than that. The scary part is no one seems to understand it or work on solving this.

Situation

Every year the competition grows linearly, with the current pandemic compounding the pressure students face now more than ever. The part of life when people start understanding different aspects of life - from what it means to tackle more responsibility to developing meaningful relationships, this growth is shadowed by the pressure to work hard in order to achieve a status in society - studying at a prestigious university and working for a billion-dollar company. Somewhere between this hustle, students forget what it means to live a life. I can vividly remember only one feeling through my years of preparing for JEE - I felt robotic, unenthusiastic, and worst of all, uninspired. There was only one thought on my mind, and that was making sure any test put in front of me, I had to be in the top 1 percentile, which is ironic because being in the top percentile is not in your control, and yet you’re judged based on how good or bad everyone else has performed relative to you. This had changed my personality significantly - I can barely remember if I ever smiled freely without being reminded of the high stakes every day. I can only be grateful for the fact that I would’ve felt worse if I was a teenager now. With this mindset, one continues to lose their sense of being human - the ability to empathize, inspire, critical thinking, etc., and turns towards making life a monotonic passage where only death can end their misery. The reality of the current educational society is this:

In the two decades of life that shape the human, we’re never once asked to pause and reflect on what it means to be human and what it means for each and everyone to live a life. Living a goal oriented life only leads to a binary outcome - a temporary satisfaction of success or a lifelong reminder of the failure.

Good Rant. Any solutions?

Frankly, I’m not sure about the solutions myself, at least at an individual level. Society is headed towards a dark path, with more and more people working toward a status rather than teaching human values. I hypothesize that human values are slowly fading away in the current society not because they’re not being taught(they never have been), but rather life during the time that’s supposed to shape our humanity is being repurposed on trying to achieve something that’s temporary.

The solution in my opinion is to influence our immediate neighborhood and hope that it has a snowball effect. If you’re a parent reading this, ask yourself: when is the last time you’ve asked your kids about what they think a happy life means(irrespective of the age), what it means to be excited about something(ice cream is not a valid topic), what does it mean to empathize with someone. If you’re a student reading this, ask yourself: when is the last time have you’ve had an original thought. No one in our society incentivizes an original thought, ironically the thing which makes us stand out in a crowd is being oppressed in the hopes that we’ll stand out later in the future. If you’re a teacher reading this, I apologize but I’m out of ideas - that’s how helpless I feel thinking about the current scenario. I’ll probably update this post once I have some clarity, but until then this has been my thoughts.

Let me know how your experiences from a point of life where societal pressure dictated your life affect you now.

I’ve been slightly regular on the Twitch platform recently, mostly to catch my favorite chess streamers Eric Rosen and Daniel Naroditsky. When I see the number of people viewing Twitch just from the list on the side, I was quite surprised. Apparently, the daily average of people on Twitch is 15 million, which is a staggering number. This made me wonder, what’s next for the researchers and engineers who build models to emulate human characteristics - WaveNet + John Legend + Google + DeepMind, Human-like chess from this paper, etc. This is where I believe Twitch can come into play.

The current state-of-the-art methods in each of the subdomains required to emulate human streaming on Twitch is covered - there are RL agents that play games, GANs that generate humans/characters(VTubers), chatbots that generate text, near-perfect text-to-speech engines. Frankly, a couple of engineers passionate about combining the current SOTA models can generate a system that can stream and try to keep the audience entertained, at least for a few minutes. This can potentially create a new benchmark on the human-level tasks, where the models are expected to perform well on the current benchmarks and are expected to work well in the pipeline with other models. Achieving this could be similar to how Tesla currently trains their networks(check out this video), but with different domains, although all these domains don’t need to be combined at once.

This is ambitious in a lot of ways, but an incremental approach toward achieving this could possibly unlock newer models, techniques, etc. Starting from a simple GAN and chatbot + text-to-speech, where the model could be fine-tuned to talk on a random-topic for the day and building this via a bottom-up approach. This is certainly exciting to think about, and by no means am I an expert on these, so if you have thoughts on this topic, feel free to comment and we can discuss!