Zone of Proximal Development

The best times of productivity often happen when in a state of “flow” – say a sustained state of seamless productivity while coding, a burst of self-expression while composing music, a sublime feeling of sentient exhilaration while working out and more. However, as great as that state is, its become increasingly hard over the years to achieve for me with all the various dopamine rollercoaster of distractions and the overwhelming demands of time. I came across constructivist methods while researching active learning methods and Vygotsky is a name that is prominent in this subject.

ZPD was a theory proposed by Lev Vygotsky in the 1930s. The crux of his proposal was that giving children experiences that better supported their learning enhanced their development. The 1930s were an interesting time where there was much debate about the the best methods of education and development for children. Vygotsky introduced the concept of ZPD to criticize the psychometric-based testing in Russian schools. He argued that academic tests where students displayed the same academic level were not the best way to determine intelligence. Instead, he focused on the augmented ability of children via social interaction to solve problems via interacting with more knowledgeable persons. He also inferred that while self-directed curiosity based approached worked for some subjects like languages , their learning in other subjects like Mathematics benefited by interacting with more “knowledgeable” folks.

Essentially ZPD is the place where once cannot progress in their learning without the interaction of a person who is high skilled at that learning objective. In a group setting, if some individuals grasp the concept, while other individuals are still in ZPD, the peer interaction between may create the most conducive environment for learning. While it makes sense at first glance, the nuance here is the ability to recognize and set learning objectives that are within the range of ZPD to enhance learning as well as ensure the right subject matter experts around in order to help complete that objective. ZPD can be applicable to various situations, everything ranging from the learning patterns of my 8-year old to setting appropriate goals to someone on my team for their advancement.

More interestingly, reflecting upon my learning blocks over the years, I have noticed multiple instances when serendipitous encounters/conversations have unblocked what seemed an impasse. It may not be too much of a stretch to apply this to religious philosophies like say Buddhism where the role of a teacher important for attainment  of one’s own “inner guru” to become a guru.

ZPD and leading teams

One of the key areas for being successful leaders is goal setting for your teams. Analyzing the tasks that are in ZPD for your team with a view to their future aspirations is invaluable.

Before goal setting, its worth the time to analyze the tasks that are within ZPD.  Then validate if the team has a variety of skill sets and expertise that can help mentorship for team members that are within ZPD but need that extra expertise to help them through it. For example, if a team member is tasked with an important large scale organizational objective, pair them with a partner that is more experienced to help them accomplish that task.

Delegating your own tasks to folks who are interested in being managers and being the  subject matter expert that helps them navigate ZPD. Challenging oneself to think about the types of learning we might or might not be doing on a daily basis and the effect of ZPD can help enhance that learning process.

Contemplating on my own learning, as it currently stands, is a daily dance around various concepts based on time, interest and compulsions – the patchwork quilt of topics consumed (tech, news etc) could use a recipe with ZPD in mind on topics that need to be reenforced or are important to enhance.

Interesting reads:

Learning Computer Science in the “Comfort Zone of Proximal Development”

Great paper on Vygotsky’s Zone of Proximal Development: Instructional Implications and Teachers’ Professional Development

Merkle Trees for Comparisons – Example

Merkle trees (named after Ralph Merkle, one of the fathers of modern cryptography) are fascinating data structures used in hash based data structures to verify the integrity of data in peer-to-peer systems. Systems like Dynamo use this to compare hashes  – essentially itself a binary tree of hashes and typically used to remove conflicts for reads. For example – in a distributed system, if a replica node falls considerably behind  its peers, using techniques like vector clocks might take unacceptable times to resolve. A hash-based comparison approach like Merkle tree would help quickly compare two copies of a range of data on different replicas. This is also a core part of blockchains like Ethereum which uses a non-binary variant but the binary ones are the most common and easy to understand and fun to implement.

Conceptually this involves:

  1. Comparing the root hashes of both trees.
  2. Continue recursion on the left and right children of the tree until the root hashes are equal.

The “Merkle root” stores the summary of all the transaction value in a singular value.

Simple Example

For example , if TA, TB,TC ,TD are transactions ( could be files, keys etc) and H is a Hash function. You can construct a tree by taking the transactions, hashing their concatenated values to generate children and finally reduced to a single root. In my scrawl above, this means hashing TA and TB, TC and TD, then hashing their concatenations H (AB), H(CD) to land at H(ABCD).Essentially keep hashing the until all the transactions meet at a single hash.

Example

Here’s an example that uses this technique to compare two files by generating their Merkle root to validate if they are equal of not (comments inline).

Invoke the script by calling “python merkle_sample.py “<file1>.csv” “<file2>.csv” to compare two merkle trees. Code below:

Key advantage here is that each branch of the tree can be checked independently without downloading the whole dataset to compare.

This translates to reducing the number of disk reads for synchronization though that efficiency needs to be balanced against the recalculation of the entire tree when nodes leave or go down. This is fundamental to Crypto currencies when transactions need to be validated by nodes and there is enormous time and space cost to validate every transaction which can be mitigated by Merkle trees in logarithmic time instead of linear time.  The Merkle root get put into the block header that gets hashed in the process of mining and comparisons are made via the Merkle root rather than submitting all the transactions over the network. Ethereum uses a more complex variant of the Merkle, namely the Merkle Patricia tree.

The applications of this range beyond blockchains to Torrents, Git, Certificates and more.

Prisoners of Geography – Book Review

Prisoners of Geography: Ten Maps That Explain Everything About the World by  Tim Marshall is an enjoyable read that traces the worlds geography  and its impact on today’s geopolitics. I picked this up on a whim as the description seemed to indicate an enjoyable refresher on the state of the world as it came to be in terms of geopolitics. The maps were pretty awful to research on my kindle paperwhite and I had to resort to getting the paperback to navigate the maps better.


A lot of the worlds problems today remain as gridlocked as they were at the origin despite decades of evolution and political talks ( think border and territory conflicts across the globe – Israel/Palestine, South China sea, South America and the list goes on). This book is not a comprehensive treatise on the evolution of those problems but a great overview. It does a good job in identifying the factors that drive the national interest and conflict in these areas and the impact of the countries dealing with the limitations/opportunities that their geography has bestowed them with.

I never did realize the importance of navigable and intersecting rivers or natural harbors that impact the destinies of countries. Here are some of my observations and notes from the book.

  • Russia – The books kicks off with the author highlighting the 100-year forward thinking of the Russians and the obsession with “warm water” ports with direct access to the ocean unlike the ports on the Arctic like Murmansk that freeze for several months. This limits the Russian fleet and its aspiration to be a bigger global power. While the oil and gas and being the 2nd biggest supplier of natural gas  in the world brings its own geographical advantages and prop the country up, its aspiration remain for fast maneuvering  to move out of areas like the Black Sea or even the Baltic Sea to counter a feared NATO strike. The author describes moves like the annexation of Crimea to be moves to construct more naval ports to boost its fleets. Countries like Moldova and Georgia ( and propensity to the west)  have a huge bearing on foreign policy and military planning. Interestingly , ‘Bear ‘is a Russian word, but per the author the Russians are also wary of calling this animal by its name, fearful of conjuring up its darker side. They call it medved, “the one who likes honey.”

It doesn’t matter if the ideology of those in control is czarist, Communist, or crony capitalist—the ports still freeze, and the North European Plain is still flat.

  • China –  The Chinese civilization, over 4000 years old that originated around the Yangtze river  is today comprised of 90% Han people united by ethnicity and politics. This sense of identity pervades all aspect of modern Chinese life and powers its  ascent as a global power. The massive Chinese border touches Mongolia in the North, Russia,Vietnam, Laos in the East and  India, Pakistan, Afghanistan and Tajikistan in the West with various levels of protracted conflicts/disagreements. For example – The India/China border is perceived by China as the Tibetan-Indian border and integral to protect the Tibetan plateau which could open a route for an Indian military push into the Chinese heartland never mind the low probability of that ever happening. The book provides a brief overview of the origin of the Tibet occupation and the worlds attention to it. The author says that if the population  were to be give a free vote, the unity of the Han would crack and weaken the hold of the communist party.  The need for  China to extend its borders and grab land it perceives as its own also extend to the seas. The growing naval fleet and its need to assert supremacy in the south china sea also fuels conflict with Japan and its neighbors. Scouring the length and breadth of Africa for minerals and precious metals in return for cheap capital and modern form of debt slavery is another strategy to dominate the world. This part of the book did not offer any new insight however a society that  holds unity and economic progress as the highest priorities is definitely admirable considering the “developing” status that it once had.
  • Unites States –  The geographical position of invulnerability, fertile land, navigable river systems and the unification of the states ensures prosperity and greatness for the U.S. The author goes into the evolution of the states as they came together after the revolutionary war such as the Louisiana purchase, the ceding of Florida by the Spanish, the Mexican war to acquire Mexico and the purchase of Alaska.  Post world war 2 and the Marshall Plan, the formation of NATO then assured the US of being the greatest firepower across the world.The author deems the Russian threat largely seen off and insists China is the rising power that the US is concerned about (as the current geopolitical climate in 2021 validates). The domination of the sea-lanes will occupy the attention with numerous potential flashpoints. Self-sufficient in energy will continue to America’s position as the preeminent economic power. Overall, this section was well summarized with the progress of American domination despite hiccups over the centuries like the great depression. I still think the author painted a rosier picture than the current situation suggests. Post-pandemic, it remains to be seen if these assertions still hold with all the internal struggles faced in the American society with respect to race relations, inclusivity and attention to a wide variety of social issues.

The California gold rush of 1848–49 helped, but the immigrants were heading west anyway; after all, there was a continental empire to build, and as it developed, more immigrants followed. The Homestead Act of 1862 awarded 160 acres of federally owned land to anyone who farmed it for five years and paid a small fee. If you were a poor man from Germany, Scandinavia, or Italy, why go to Latin America and be a serf, when you could go to the United States and be a free land-owning man?

  • Western Europe – Again, the geographical blessings in this case ensured an agreeable climate mostly to cultivate the right crops at large scale, the right minerals to power the industrial revolution and abundant natural harbors. This led to industrial scale wars as well as Europe remains an amalgam of linguistic and culturally disparate countries yet remains an industrial power. The contrast between northern and southern Europe in terms of prosperity is attributed to industrialization, the domination of Catholicism, and the availability of coastal plains. Spain, Greece, U.K, Germany, Poland, Denmark, Sweden  and the contrasts in their economical status are discussed and  attributed to geographical limitations.  I was hoping the  author provides more than a passing nod to the concerns of immigration and prejudice. Prejudice against immigration and the rise of nationalism remains on the rise across the world and its troubling to see this rise of hate groups, holocaust deniers and all other abhorrent tribes that debase basic human ideals of equality, peace and harmony.  The demographic change with the inverted pyramid of older people at the top with fewer people paying taxes to support them in the future needs to be reversed and the benefits of legal immigration need to be given greater attention rather than burying them under misdirected xenophobic fears.
  • Africa – This was an enlightening section on the lack of utility of African rivers for transportation due to waterfalls and natural obstacles. Africa developed in isolation from the Eurasian landmass and the author asserts that the lack of idea exchange played a huge part in its under development. Sub-saharan exposure to virulent diseases, crowded living conditions and poor health-care infrastructure has also impeded growth. The great rivers of Africa—the Niger, the Congo, the Zambezi, the Nile, and others—don’t connect to its own detriment. The 56~ countries have relatively unchanged borders over the years along with the legacy of colonialism which like most parts of the world divided societies on the basis of ethnicities. The rise of radical Islamist groups has been attributed to the sense of underdevelopment and overall malcontent. On a more positive note, every year roads and railroads are fueling infrastructure boom and greater connectivity with rising education and healthcare.

“You could fit the United States, Greenland, India, China, Spain, France, Germany, and the UK into Africa and still have room for most of Eastern Europe. We know Africa is a massive landmass, but the maps rarely tell us how massive.”

  • Middle East – Another witness to the ancient civilization that rose from the fertile plains of Mesopotamia. The largest continuous sand desert that  British and French colonists carved up as part of the Sykes-Picot carving reflects some of the unrest and extremism today. Its interesting that prior to Sykes-Picot, there was no Syria, Lebanon,  Jordan, Iraq, Saudi Arabia, Kuwait, Israel, or Palestine. These are all modern entities with a short history unified by versions of the same religion. Conflict and chaos have ruled supreme in some of the countries ( Iraq, Lebanon etc) while prosperity from the oil fields have propelled some to the world stage (UAE). Lot of detail on Iran-Iraq history, Palestine, the failed promise of the Arab spring, Turkey and others. The complexity of the demographics and religious idealism compounds an already volatile region.

Sykes-Picot is breaking; putting it back together, even in a different shape, will be a long and bloody affair.

  • India & Pakistan– A population of 1.4 Billion pitted against another of 182 million with impoverishment, volatility and mistrust at both ends. Post-pandemic, this section is dated as the imminent Indian emerging economic power described by the author is no longer a reality at least in the near term. Had to skip over this section as there wasn’t much I didn’t already know.
  • Korea and Japan– Tension between the Koreas is well known to the world and the author describes the origins of the Hermit kingdom and the lack of strategy from the USA in dealing with the problem. The 38th parallel was yet another hasty line of division and an uninformed repetition of the line drawn in the aftermath of the Ruso-Japanese war of 1904. I have fond memories of visiting Seoul years ago and it was interesting on how the concept of unification was welcomes by some of the South Koreans I had the opportunity to interact with (peering through the binoculars in the DMZ to the North Korean side was a thrilling experience and emphasized the proximity of the two sides). Not sure if that is the general sentiment but there is enough justification there considering the Northern nuclear power in control of a dictator. The Japanese post-war stance is described in detail and the author contends the increasing Japanese defense budget displays the intent of resolve against Chinese threats.
  • Latin America– Limitation of the Latin America originates from the historical inequality, the reluctance of the original settles to move away from the coats and the lack of subsequent infrastructure in the interiors. Geographical limitations plague Mexico, Brazil, Chile, Argentina despite natural resources. The civil wars of the 19th century broke apart independent countries with border disputes that persist, naval arms races between countries like Brazil, Argentina and Chile held back development of all three and drug cartels have devastated societies. The Panama canal’s newer rival – the Nicaragua Grand Canal that has a huge Chinese investment across the continent seem questionable in terms of value to Latin America.
  • The Arctic– The effects of global warming are alarmingly showing in the Arctic coinciding with the discovery of energy deposits. The complex land ownership includes land in parts of Canada, Finland, Greenland, Iceland, Norway, Russia, Sweden, and the United States (Alaska). The melting ice has far flung ramifications globally in terms of projected flooding effects in countries far away as Maldives. The melting ice has also opened up new transportation corridor that hugs the Siberian coastline and more access to energy reserves much to the interest of multiple countries that are now jostling for superiority including Russia building an Arctic army.

The word arctic comes from the Greek arktikos, which means “near the bear,” and is a reference to the Ursa Major constellation, whose last two stars point toward the North Star.

A lot of rich detail in the book on various nuances of the geographies and this was an enjoyable read however it did make me pessimistic as status quo or deterioration of the situation in a lot of these geographies has been the norm. As the 21st century progresses, there is not much indication that change is afoot unless a planet threatening situation like climate change becomes a forcing function to minimize petty squabbles to focus on larger resolutions. Being Idealistic or moralistic will not jive well with the ideas in this book and the way forward is to think of creative and new ideas to resolve a lot of these global problems. Great ideas and great leaders need to arise to challenge these realities and put humanity first.

As the author ruefully writes:

” A human being first burst through the top layer of the stratosphere in 1961 when twenty-seven-year-old Soviet cosmonaut Yuri Gagarin made it into space aboard Vostok 1. It is a sad reflection on humanity that the name of a fellow Russian named Kalashnikov is far better known.”


A couple of other reads recommended to me are Peter Zeihan’s “The Accidental Superpower” and Robert D Kaplan’s “The Revenge of Geography”. I look forward to reading them as well.

Text Summarizer on Hugging Face with mlflow

Hugging Face Emoji Classic Round Sticker - EmojiPrints

Hugging Face is the go-to resource open source natural language processing these days. The Hugging Face hubs are an amazing collection of models, datasets and metrics to get NLP workflows going. Its relatively easy to incorporate this into a mlflow paradigm if using mlflow for your model management lifecycle. mlflow makes it trivial to track model lifecycle, including experimentation, reproducibility, and deployment. mlflow’s open format makes it my go-to framework for tracking models in an array of personal projects and It also has an impressive enterprise implementation that my teams at work enable for large enterprise use cases. For smaller projects, its great to use mlflow locally for any projects that requires model management as this example illustrates.

The beauty of Hugging Face (HF) is the ability to use their pipelines to to use models for inference. The models are products of massive training workflows performed by big tech and available to ordinary users who can use them for inference. The HF pipelines offer a simple API dedicated to performing inference in these models thus sparing the ordinary the user the complexity and compute / storage requirements for running such large models.

The goal was to put some sort of tracking around all my experiments with the Hugging Face Summarizer that I’ve been using to  summarize text and then use the mlflow Serving via REST as well as running predictions on the inferred model by passing in a  text file. Code repository is here with snippets below.

Running the Text Summarizer and calling it via curl

Text summarization consists of Extractive and Abstractive types where Extractive selects sentence that has the most valuable context while Abstractive is trained to create summaries.

Considering I was running on a CPU, I picked a small model like the T5-small model trained on Wikihow All data set that has been trained to write summaries. The boiler plate code on the HuggingFace website gives you all you need to get started. Note that this models input length is set to 512 tokens max which may not be optimum for usecases with larger text.

a) First step is to define a wrapper around the model code so it can be called easily later on by subclassing it with the mlflow.pyfunc.PythonModel to use custom logic and artifacts.

class Summarizer(mlflow.pyfunc.PythonModel):
    '''
    Any MLflow Python model is expected to be loadable as a python_function model.
    '''

    def __init__(self):
        from transformers import pipeline, AutoTokenizer, AutoModelWithLMHead

        self.tokenizer = AutoTokenizer.from_pretrained(
            "deep-learning-analytics/wikihow-t5-small")

        self.summarize = AutoModelWithLMHead.from_pretrained(
            "deep-learning-analytics/wikihow-t5-small")

    def summarize_article(self, row):
        tokenized_text = self.tokenizer.encode(row[0], return_tensors="pt")

        # T5-small model trained on Wikihow All data set.
        # model was trained for 3 epochs using a batch size of 16 and learning rate of 3e-4.
        # Max_input_lngth is set as 512 and max_output_length is 150.
        s = self.summarize.generate(
            tokenized_text,
            max_length=150,
            num_beams=2,
            repetition_penalty=2.5,
            length_penalty=1.0,
            early_stopping=True)

        s = self.tokenizer.decode(s[0], skip_special_tokens=True)
        return [s]

    def predict(self, context, model_input):
        model_input[['name']] = model_input.apply(
            self.summarize_article)

        return model_input

b) We define the tokenizer to prepare the inputs of the model and the model using the HuggingFace specifications. This is a smaller model trained on Wikihow All data set. From the documentation – the model was trained for 3 epochs using a batch size of 16 and learning rate of 3e-4. Max_input_length is set as 512 and max_output_length is 150.

c) Then define the model specifications of the T5-small model by calling the summarize_article function with the tokenized text that will called it for every row in the dataframe input and eventually return the prediction.

d) The prediction function calls the summarize_article providing the  model input and calling the summarizer and returns the prediction. This is also where we can plug in mlflow  to infer the predictions.

The input and output schema are defined in the ModelSignature class as follows :

# Input and Output formats
input = json.dumps([{'name': 'text', 'type': 'string'}])
output = json.dumps([{'name': 'text', 'type': 'string'}])
# Load model from spec
signature = ModelSignature.from_dict({'inputs': input, 'outputs': output}) input = json.dumps([{'name': 'text', 'type': 'string'}])
 output = json.dumps([{'name':'text', 'type':'string'}]) 


e) We can set mlflow operations by setting the tracking URI which was “” in this case since its running locally. Its trivial in a platform like Azure to spin up a databricks workspace and get a tracking server spun up automatically so you can persist all artifacts at cloud scale.


Start tracking the runs by wrapping the mlflow.start_run invocation. The key here is to call the model for inference using the mlflow.pyfunc function to make the python code load into mlflow. In this case , the dependencies of the model are all stored directly with the model. Plenty of parameters here that can be tweaked described here.

# Start tracking
with mlflow.start_run(run_name="hf_summarizer") as run:
    print(run.info.run_id)
    runner = run.info.run_id
    print("mlflow models serve -m runs:/" +
          run.info.run_id + "/model --no-conda")
    mlflow.pyfunc.log_model('model', loader_module=None, data_path=None, code_path=None,
                            conda_env=None, python_model=Summarizer(),
                            artifacts=None, registered_model_name=None, signature=signature,
                            input_example=None, await_registration_for=0)


f) Check the runs via mlflow UI either using the “mlflow ui” command or just invoke the commandmlflow models serve -m runs:/<run_id>


g) Thats it – Call the curl command using sample text below:

curl -X POST -H "Content-Type:application/json; format=pandas-split" --data '{"columns":["text"],"data":[["Howard Phillips Lovecraft August 20, 1890 – March 15, 1937) was an American writer of weird and horror fiction, who is known for his creation of what became the Cthulhu Mythos.Born in Providence, Rhode Island, Lovecraft spent most of his life in New England. He was born into affluence, but his familys wealth dissipated soon after the death of his grandfather. In 1913, he wrote a critical letter to a pulp magazine that ultimately led to his involvement in pulp fiction.H.P.Lovecraft wrote his best books in Masachusettes."]]}' http://127.0.0.1:5000/invocations

Output:

"name": "Know that Howard Phillips Lovecraft (H.P.Lovecraft was born in New England."}]%

Running the Text Summarizer and calling it via a text file

For larger text, its more convenient reading the text from a file, formatting it and running the summarizer on it. The predict_text.py does exactly that.


a) Clean up the text in article.txt and load the text into a dictionary.

b) Load the model using pyfunc.load_model and then run the model.predict on the dictionary.

# Load model as a PyFuncModel.
loaded_model = mlflow.pyfunc.load_model(logged_model)


# Predict on a Pandas DataFrame.
summary = loaded_model.predict(pd.DataFrame(dict1, index=[0]))

print(summary['name'][0])

Code here

In summary, this makes for a useful way to track models and outcomes from readily available transformer pipelines to pick the best ones for the task.

Spotify Recommender API Call

One of my favorite features in Spotify are the recommendations. The app’s recommendations includes the Discover Weekly,  Daily Mix, Release Radar and the Artist Radio features. I could go through hours of recommendations substituting white noise while working on projects and usually encounter a song or an artist that appeals to my guitar/keyboard driven sensibilities in a session. While Discover Weekly, Daily Mix yield gems once in a while, the song specific ones usually based on Artist / Song radio yield a lot more matches to my sensibilities.

The recommendations endpoints that generates reccs based on a seed is a favorite. I’ve usually had a good match rate with songs that “stick” based on the API. There are plenty of other endpoints (artists, songs  etc) that could be easily plugged in to generate relevant predictions.

The API documentation of Spotify has always been stellar and its usability is enhanced by being able to test all the API calls easily within their developer console.

This API also has a bunch of parameters that can be configured for fine-tuning the recommendation: key, genre, loudness, energy, instrumentalness, popularity, speechiness, danceability etc.

Per the official docs – “Recommendations are generated based on the available information for a given seed entity and matched against similar artists and tracks. If there is sufficient information about the provided seeds, a list of tracks will be returned together with pool size details. For artists and tracks that are very new or obscure there might not be enough data to generate a list of tracks.

One of the key things here is to generate seeds for the recommendations, this can be done by using endpoints like Get a User’s Top Artists and Tracks to obtain artists and tracks based on my listening history and use these artists and tracks as seeds for the Get Recommendations Based on Seeds endpoint. This endpoint will only return tracks. 
The Web API Authorization Guide is a must to read before querying these endpoints and the developer console makes it super easy to try out different endpoints. 

I wanted a quick way to query the recommendations API for new recommendations and the combination of the streamlit + Spotify API was quick simple solve to get that working. At a high level I wanted to be able to query a song or artist and generate recommendations based on it. A secondary need is also to collect data for a reccomender I am training to customize ML-driven reccomendations but more on that in a different post.

A lot of the code is boilerplate and pretty self explanatory but at a high level it consists of the class to interact with the Spotify API (spotify_api.py) ,  a UI wrapper using Streamlit to render the app (spotify_explorer.py).  Given a client id and client secret, spotify_api.py gets client credentials from the Spotify API to invoke the search. Sample code inline with comments. The code can obviously be much more modular and pythonic but for investing a quick hour of hacking, this got the job done.

class SpotifyAPI(object):
    access_token = None
    access_token_expires = datetime.datetime.now()
    access_token_did_expire = True
    client_id = None
    client_secret = None
    token_url = 'https://accounts.spotify.com/api/token'

    def __init__(self, client_id, client_secret, *args, **kwargs):
        self.client_id = client_id
        self.client_secret = client_secret

    # Given a client id and client secret, gets client credentials from the Spotify API.
    def get_client_credentials(self):
        ''' Returns a base64 encoded string '''
        client_id = self.client_id
        client_secret = self.client_secret
        if client_secret == None or client_id == None:
            raise Exception("check client IDs")
        client_creds = f"{client_id}:{client_secret}"
        client_creds_b64 = base64.b64encode(client_creds.encode())
        return client_creds_b64.decode()

    def get_token_header(self):  # Get header
        client_creds_b64 = self.get_client_credentials()
        return {"Authorization": f"Basic {client_creds_b64}"}

    def get_token_data(self):  # Get token
        return {
            "grant_type": "client_credentials"
        }

    def perform_auth(self):  # perform auth only if access token has expired
        token_url = self.token_url
        token_data = self.get_token_data()
        token_headers = self.get_token_header()

        r = requests.post(token_url, data=token_data, headers=token_headers)

        if r.status_code not in range(200, 299):
            print("Could not authenticate client")
        data = r.json()
        now = datetime.datetime.now()
        access_token = data["access_token"]
        expires_in = data['expires_in']
        expires = now + datetime.timedelta(seconds=expires_in)
        self.access_token = access_token
        self.access_token_expires = expires
        self.access_token_did_expire = expires < now
        return True

    def get_access_token(self):

        token = self.access_token
        expires = self.access_token_expires
        now = datetime.datetime.now()
        if expires < now:
            self.perform_auth()
            return self.get_access_token()
        elif token == None:
            self.perform_auth()
            return self.get_access_token()
        return token

    # search for an artist/track based on a search type provided
    def search(self, query, search_type="artist"):
        access_token = self.get_access_token()
        headers = {"Content-Type": "application/json",
                   "Authorization": f"Bearer { access_token}"}
        # using the  search API at https://developer.spotify.com/documentation/web-api/reference/search/search/
        search_url = "https://api.spotify.com/v1/search?"
        data = {"q": query, "type": search_type.lower()}
        from urllib.parse import urlencode
        search_url_formatted = urlencode(data)
        search_r = requests.get(
            search_url+search_url_formatted, headers=headers)
        if search_r.status_code not in range(200, 299):
            print("Encountered isse=ue")
            return search_r.json()
        return search_r.json()

    def get_meta(self, query, search_type="track"):  # meta data of a track
        resp = self.search(query, search_type)
        all = []
        for i in range(len(resp['tracks']['items'])):
            track_name = resp['tracks']['items'][i]['name']
            track_id = resp['tracks']['items'][i]['id']
            artist_name = resp['tracks']['items'][i]['artists'][0]['name']
            artist_id = resp['tracks']['items'][i]['artists'][0]['id']
            album_name = resp['tracks']['items'][i]['album']['name']
            images = resp['tracks']['items'][i]['album']['images'][0]['url']

            raw = [track_name, track_id, artist_name, artist_id, images]
            all.append(raw)

        return all

  

The get_recommended_songs function is the core of the app querying the API for results based on the query passed in. The more the parameters the better the results. Customizing the call to any API call is fairly trivial.

   def get_reccomended_songs(self, limit=5, seed_artists='', seed_tracks='', market="US",
                              seed_genres="rock", target_danceability=0.1):  # reccomendations API
        access_token = self.get_access_token()
        endpoint_url = "https://api.spotify.com/v1/recommendations?"
        all_recs = []
        self.limit = limit
        self.seed_artists = seed_artists
        self.seed_tracks = seed_tracks
        self.market = market
        self.seed_genres = seed_genres
        self.target_danceability = target_danceability

        # API query plus some additions
        query = f'{endpoint_url}limit={limit}&market={market}&seed_genres={seed_genres}&target_danceability={target_danceability}'
        query += f'&seed_artists={seed_artists}'
        query += f'&seed_tracks={seed_tracks}'
        response = requests.get(query, headers={
                                "Content-type": "application/json", "Authorization": f"Bearer {access_token}"})
        json_response = response.json()

        # print(json_response)
        if response:
            print("Reccomended songs")
            for i, j in enumerate(json_response['tracks']):
                track_name = j['name']
                artist_name = j['artists'][0]['name']
                link = j['artists'][0]['external_urls']['spotify']

                print(f"{i+1}) \"{j['name']}\" by {j['artists'][0]['name']}")
                reccs = [track_name, artist_name, link]
                all_recs.append(reccs)
            return all_recs

Wrapping both the calls in a Streamlist app is refreshingly simple and dockerizing and pushing to Azure container registry was trivial.

Code

https://github.com/vishwanath79/spotifier

Usage

To run the app, run:
streamlit run spotify_explorer.py

Deployed at

https://spotiapp2.azurewebsites.net/

Part 2 to follow at some point as I continue building out a custom recommender that compares the current personalizer with a custom personalizer that takes in Audio features and more personalized inputs and tuneable parameters.

2020 in Books

Booklist

I usually set reading goals at the beginning of the year to counter the array of distractions that have dented my reading habit over the last few decades. goodreads is great for maintaining some accountability and I’m in awe of some of my friends who seem to knock out 80-100 books easily every year despite their hectic work lives. I tried to veer myself away from my usual mix of management/leadership books this year to reignite fiction reading. The results were mixed thanks to massive mood swings through the year and I ended up with a random mix of tech, biographies, music, non-fiction and horror/sci-fi tomes that helped distract me from a depressing year.

Complete list of books here.

Some notable ( and not so notable) reads this year:

Greetings from Bury Park by Sarfraz Manzoor: A Springsteen tribute along with an immigrant experience? Sign me up! After enjoying the Blinded by the night with my better half, I had to get my hands on Sarfraz Manzoor’s ode to the boss amidst the backdrop of immigrant life in 80s Luton. The book is set in a “non-linear” timeline mode which may put off some readers but I found it a wonderful read albeit with some cliched moments probably dramatized. It also took me to pilgrimage of the boss’ older catalog and some of his late 80s/early 90s work that I love.(Think Tunnel of Love/ Human Touch era). As a huge music fan and an immigrant who traveled halfway around the world to chase dreams built on the foundational goals of exposing myself to new cultures/thinking and seeing my favorite musicians in live arenas in the flesh,the book resonated with me on the universality of music across cultures.

The Billionaire Raj: A Journey Through India’s New Gilded Age: India is a land of skewed levels of haves and have nots at an unprecedented scale. You don’t understand it till you see it and even when you see it, your understanding is peripheral at best when confronted with the magnitude of the problem and the inequality at scale. James Crabtree’s detailed tome on the lifestyles of the rich and famous in India helps decrypt the inequalities and the “crony capitalism” that ensures the system stays that way. The nexus between politics, Bollywood, business tycoons are all deciphered out and connected together to explain the irony of situations like a ~2 Billion dollar personal home in Mumbai towering over a squalor of a million people in a nearby slum.

Tesseract by Alex Garland : I’m a big fan of Alex Garland’s works – 28 Days Later, The Beach, Sunshine and Ex-machina. On top of it all, he slam dunked the best version of Dredd on us innocent fans and immortalized Keith Urban in that role. I had huge hopes for Tesseract but it ended up being a random story of disparate characters linked together by the thinnest of chances and a shallow plot. Not his finest hour and I could not wait to finish this as it felt as slow as  drug-induced slow-motion sequences in Dredd which were way more enjoyable. Still a huge AG fan regardless.

Seven years in Tibet by Heinrich Harrer: Heinrich Harrer’s discovery of Tibet deserves it own post.

Devolution by Max Brooks: Anyone who has read World War Z knows the doom-laden nightmarish scenarios that the author can generate and this one is no exception. Obnoxious characters dealing with first-world problems in their isolated eco-friendly community encounter an even more ominous situation with Mt.Rainier erupting and get blockaded. Hell breaks loose in the form of rampaging sasquatches who ( thankfully in some cases) start taking out the characters one by one. A personal journal left behind serves as the narrative and tons of interesting sasquatch legends abound including this one of Roosevelt’s own” encounter” with the sasquatch.

The Long Walk by Stephen King: Big fan of the king. The plot and premise was great here but it did slow down towards the end. On hindsight, this book is probably best enjoyed via an audio book while on the treadmill. Overall an OK read.

Ibn Fadlan and the Land of Darkness: Arab Travellers in the Far North: Travel log of Ibn Fadlan , a tenth-century diplomat who, in 922 AD, was sent on a mission from Baghdad to the far north by the caliph Muqtadir. His journal serves as an important account of life in mordern-day Russia/Middle east and the areas in between. Repetitive and slow moving in places , the book abounds with interesting details on the trading routes, strange customs, vikings raids, savage rituals, food habits, wealth management, religion and a disconnected world at a strange dark time.

Bloodcurdling Tales of Horror and the Macabre: The Best of H. P. Lovecraft by H.P. Lovecraft : Lovecraft (despite his controversial views) is the master of horror fiction and I’ve always enjoyed the predictable nature of a classic lovecraft pulp. More so for the constant reminders that much of what we know is unknown and we are all inconsequent specks of dust in the vastness of time ( To somewhat paraphrase Carl Sagan, if the entire history of earth was compressed into 365 days, humans would have existed for ~30 seconds.) Usually a Halloween ritual, this year was no different with Lovecraft to distract me from the horrors raging outside. This collection has all the usual ones including Call of Cthulhu, Dunwich Horror and the Shadow over Innsmouth.

Idea Makers: Personal Perspectives on the Lives & Ideas of Some Notable People by Stephen Wolfram : Short biographies of giants in the filed of mathematics/computer science. Despite the authors inclination to insert the use of Mathematica tool into a “what-if” scenario into every biography, this was still an enjoyable read. The author is a giant in his field and I suppose some level of braggadocio is expected. The fascinating backstories into characters like Leibnz, Babbage, Feynman and Ramanujan is written powerfully and by someone with the grasp of minutiae of their research areas which is awe-inspiring. My highlights here.

Tech

The tech books followed a predictable pattern of excellent reads this year helping me keep abreast of work-related topics mostly focused on Spark, Databases and ML.

Database Internals: A deep-dive into how distributed data systems work by Alex Petrov: Informative but would have preferred more examples with practical scenarios. No code and this all mostly conceptual. Some good references to papers for subsequent reading. The first part of the book deals primarily with storage and covers an in-depth discussion of b-trees and types. The second half is focused on distributed systems and has useful sections on consensus protocols. Concepts like “2-phase commits” are explained well with figures. However, the lack of practical examples/code and overall dry subject matter made this a laborious read. Good book to reference theoretical concepts. 

Practical Deep Learning for Cloud, Mobile, and Edge: Real-World AI & Computer-Vision Projects Using Python, Keras & Tensorflow: Plenty of examples and links for more research. The material is too vast enough to make an all encompassing book but this delivers in terms of practical tips. Lots of practical tips provided that will find a place in any serious ML engineer repertoire. The consolidated list of tips are worth the book alone. Excellent comparisons of Raspberry Pi, Jetson Nano, and Google Coral. The reinforcement learning sections could have used some more practical examples in areas like q-learning but overall great read and reference material.

Music

The books this year included Van Halen ( Eddie R.I.P), Megadeth, Glyn Johns amongst others.

Hard to Handle: The Life and Death of the Black Crowes–A Memoir by Steve Gorman: Great read start to finish. The Black Crowes were one of the many soundtracks of my teenage years in the 90s. The first three albums are seminal works that stand out despite having to contend with changing musical climate with the rise of grunge and the decline of hair metal. The book is a page-turner for anyone even vaguely familiar with the Black Crowes. It made me go back and re-immerse into the catalog especially with albums like By your side which is an underrated gem produced by the great Kevin Shirley and their successful but short-lived collaboration with Jimmy Page – Live at the Greek. Gorman’s insight as a founding member and the frank admission of all the dysfunction makes up for a great story. Im a huge fan of Rich Robinson’s use of open G tuning and this book has led to more inspired practicing in that vein.

Overall 35 books for the year which I can hopefully better in 2021. Current reading list here.

ONNX for ML Interoperability

Having been a Keras user since  I read  the seminal Deep Learning with Python , I’ve been experimenting with exporting formats to different frameworks to be more framework-agnostic.


ONNX ( Open Neural Network Exchange) is an open format for representing traditional and deep learning ML models.  Key goal being promoting inter-operability between a variety of frameworks and target environments. ONNX helps you to export a fully trained model into its format and enables targeting diverse environments without you doing manual optimization and painful rewrites of the models to accommodate environments.
It defines an extensible computation graph model along with built-in operators and standard data types to allow for a compact and cross-platform representation for serialization. A typical use case could be scenarios where you want to use transfer learning to use model weights of another model possibly built in another framework into your own model i.e. if you build  a model in Tensorflow, you get a protobuf (PB) file as output and it would be great if there is one universal format that you can now convert to the PT format to load and reuse in Pytorch or use its own hardware agnostic runtime.

For high-performance inference requirements in varied frameworks, this is great with platforms like NVIDIA’s TensorRT supporting ONNX with optimizations aimed at the accelerator present on their devices like the Tesla GPUs or the Jetson embedded devices.

Format

The ONNX file is a protobuf encoded tensor graph. List of operators supported are documented here and operations are referred to as “opsets” i.e. operation sets. Opsets are defined for different runtimes in order to enable interoperability. The operations are a growing list of widely used linear operations, functions and other primitives used to deal with tensors.

The operations include most of the typical deep learning primitives, linear operations, convolutions and activation functions. The model is mapped to the ONNX format by executing the model with often just random input data and tracing the execution. The operations executed are mapped to ONNX operations and so the entire model graph is mapped into the ONNX format. After this the ONNX model is then saved as .onnx protobuf file which can be read and executed by a wide and growing range of ONNX runtimes.

Note – Opsets are fast evolving and with fast release cycles of competing frameworks, it may not always be easy to upgrade to the latest ONNX version if it breaks compatibility with other frameworks. The file format consists of the following:

  • Model: Top level construct
    • Associates version Info and Metadata with a graph
  • Graph: describes a function
    • Set of metadata fields
    • List of model parameters
    • List of computation nodes – Each node has zero or more inputs and one or more outputs.
  • Nodes: used for computation
    • Name of node
    • Name of an operator that it invokes a list of named inputs
    • List of named outputs
    • List of attributes

More details here.

Runtime


The ONNX model can be inferenced with ONNX runtime that uses a variety of hardware accelerators for optimal performance. The promise of ONNX runtime is that it abstracts the underlying hardware to enable developers to use a single set of APIs for multiple deployment targets. Note – the ONNX runtime is a separate project and aims to perform inference for any prediction function converted to the ONNX format.

This has  advantages over dockerized pickle models that is usually the approach in a lot of production deployments where there are runtime restrictions (i.e. can run only in .NET or JVM) , memory and storage overhead, version dependencies, and batch prediction requirements.


ONNX runtime has been integrated in WINML, Azure ML with MSFT as its primary backer. Some of the new enhancements include INT8 quantization to reduce floating point numbers for reducing model size, memory footprint and to increase efficiencies benchmarked here.


The usual path to proceed :

  • Train models with frameworks
  • Convert into ONNX with ONNX converters
  • Use onnx-runtime to verify correctness and Inspect network structure using netron (https://netron.app/)
  • Use hardware-accelerated inference with ONNX runtime ( CPU/GPU/ASIC/FPGAs)

Tensorflow


To convert Tensorflow models, the easiest way is to use the tf2onnx tool from the command line. This converts the saved model to a model representation that includes the inference graph.


Here is an end-to-end example of saving a simple Tensorflow model , converting it to ONNX and then running the predictions using the ONNX model and verifying the predictions match.


Challenges


However, some things to consider while using this format is the lack of “official support” from frameworks like Tensorflow. For example, Pytorch does provide the functionality to exports models into ONNX (torch.ONNX ) however I could not find any function to import an ONNX model to out put a Pytorch model. Considering CAFFE 2 that is a part of PyTorch fully supports ONNX import/export, it may not be totally unreasonable to expect an official conversion importer(there is a proposal already documented here).

The Tensorflow converters seem to be part of the ONNX project i.e. not an official/out of the box Tensorflow implementation. List of Tensorflow Ops supported are documented here. The github repo is a treasure trove of information on the computation graph model and the operators/data types that power the format. However, as indicated earlier depending on the complexity of the model (especially in transfer learning scenarios), it’s likely to encounter conversion issues during function calls that may cause the ONNX converter to fail. In this case, there are likely scenarios which may necessitate modifying the graph in order to fit the format. I’ve had a few issues running into StatefulPartitionednCalls especially in using TransferLearning situations for larger encoders in language models.


I have also had to convert Tensorflow to PyTorch by first converting Tensorflow to ONNX. Then the ONNX models to Keras using onnx2keras and then convert to Pytorch using MMdn with mixed results and a lot of debugging and many abandons. However, I think ONNX runtime for inference rather than framework-to-framework conversions will be a better use of ONNX.



The overall viability of a universal format like ONNX though well intentioned and highly sought may not fully ever come into fruition with so many divergent interests amongst the major contributors and priorities though its need cannot be disputed.

Replay

Replay is a collaboration track and part of an evolving experiment with multi-tracked guitars revolving around cyclic patterns. More collaborations and sounds to follow.

Arjun on Bass

Sunder – Drums (Instagram and Facebook – @onlysunder)