Posted on

Tenacity for retries

Most times when I have taken over new roles or added more portfolios to my current workstreams, it has usually involved decision making to either build on or completely overhaul legacy code. I’ve truly encountered some horrific code bases that are no longer understandable, secure or  scaleable. Often if there is no ownership of the code or if attrition has rendered it an orphan, it usually sits unnoticed ( like an argiope spider waiting for its prey) until it breaks (usually on a friday afternoon) when someone has to take ownership of it and bid their weekend plans goodbye. Rarely do teams have  a singular coding style with patterns and practices clearly defined that are repeatable and can withstand change people or technology change – if you are in such an utopian situation, consider yourself truly lucky!

A lot of times,  while you have to understand and reengineer the existing codebase, it is imperative to keep the lights on while you are figuring out the right approach, a sorta situation while you got to keep the car moving while changing its tires. I discovered Tenacity while  encountering a bunch of  gobbledygook shell and python scripts crunched together that ran a critical data pipeline and had to figure out a quick retry logic to keep the job running  while it randomly failed due to environment issues ( exceptions, network timeouts, missing files, low memory and every other entropy-inducing situation on the platform). The library can handle different scenarios based on your usecase.

  • Continuous retrying for your commands. This usually works if its a trivial call where you have minimal concerns over overwhelming target systems. Here, the function can be retried until no exception is returned.
  • Stop after a certain number of tries. Example – if you have a shell script like shellex below that needs to execute and you anticipate delays or issues with the target URL or if a command you run raises an exception, you can have Tenacity retry based on your requirement .
  • Exponential backoff patterns – To handle cases where the delays maybe too long or too short for a fixed wait time, tenacity has a wait_exponential algo that can ensure that if a request cant succeed in a short time after a retry, the application can wait longer and longer with each retry thus alleviating the target system of repetitive fixed time retries.

The library handles plenty of others uses cases like Error handling, custom callbacks and tons more.

Overall, this has been a great find to use a functional library like Tenacity to for various usecases instead of writing custom retry logic or implementing a new orchestration engine for handling retries.

Posted on

Book Review – Machine Learning with PyTorch and Scikit-Learn

by Sebastian Raschka & Vahid Mirjalili

I primarily wanted to read this book due to the Pytorch section and pretty much flipped through the Scikit learn section while absorbing and practicing with the Pytorch section. So my review largely is based on chapter 13 and beyond. Apart from the Pytorch official documentation, there are not too many comprehensive sources that can serve as a good reference with practical examples for Pytorch in my view. This book aims to do that and pretty much hits the mark.

Ch 1-11 is a good refresher on SciKit -learn and sets up all the foundational knowledge you need for the more advanced concepts with Pytorch. I wish though the authors had different examples and not resorted to ubiquitous examples like MNIST ( like in chapter 11/ 13 etc) for explaining neural network concepts. While these are good for understanding foundational concepts, I find the really good books usually veer away from standard examples found online and get creative.

Chapter 12 provides an excellent foundation in Pytorch and a primer for building a NN model in Pytorch. The code examples are precise with the data sources clearly defined so I could follow along without any issues. I did not need a GPU/collab to run examples . Good to see the section on writing custom loss functions as those are useful practical skills to have.

Ch-14 which has us training a smile classifier to explain convolution neural networks is a useful example especially for tricks like data augmentation that can be applied to other usecases.

I skipped through the chapter on RNNs as transformers are the rage now ( Ch-16) and Pytorch already has everything implemented in its wrapper function for structures like LSTMs. Still , a lot of dense and useful material explaining the core concepts behind RNNs and some interesting text generation models using Categorical pytorch classes to draw random samples.

The chapter on Transformers is a must-read and will clear up a lot of foundational concepts. Another thing to mention is that the book has well depicted color figures that make some of the dense material more understandable. Contrasting the transformers approach to RNNs using concepts like attention mechanisms is clearly explained. More interestingly, the book dwells into building larger language models with unlabeled data such as BERT and BART. I plan to re-read this chapter to enhance my understanding of transformers and the modern libraries such as HuggingFace that they power.

The chapter of GANs was laborious with more MNIST examples and could have had a lot more practical examples.

Ch-18 on Graph Neural Network is a standout section in the book and provides useful code examples to build pytorch graphs to treat as datasets defining edges and nodes. For example, libraries like Torch Drug are mentioned that use pytorch Geometric framework for drug discovery. Spectral graph convolution layers, graph pooling layers, and
normalization layers for graphs are explained and I found this chapter to be a comprehensive summary that would save one hours of searching online for the fundamental concepts. GNNs definitely have a ton of interesting applications and a lot of recent papers with links are provided.

Ch-19 on reinforcement learning adds another dimension to the book which is largely focused on supervised and unsupervised learning in prior chapters. Dynamic programming to Monte Carlo to Temporal Difference methods are clearly articulated. The standard open AI gym examples are prescribed for implementing grids to specify actions and rewards. I thought this chapter was great explaining the theoretical concepts but the examples were all the standard Q-learning fare you would find online. Would have loved to see a more realistic example or pointers to apply to your own usecases.

All in all, I enjoyed the Pytorch examples and clearly explained concepts in this book and it would be a good Pytorch reference to add to your library.

Posted on

Merkle Trees for Comparisons – Example

Merkle trees (named after Ralph Merkle, one of the fathers of modern cryptography) are fascinating data structures used in hash based data structures to verify the integrity of data in peer-to-peer systems. Systems like Dynamo use this to compare hashes  – essentially itself a binary tree of hashes and typically used to remove conflicts for reads. For example – in a distributed system, if a replica node falls considerably behind  its peers, using techniques like vector clocks might take unacceptable times to resolve. A hash-based comparison approach like Merkle tree would help quickly compare two copies of a range of data on different replicas. This is also a core part of blockchains like Ethereum which uses a non-binary variant but the binary ones are the most common and easy to understand and fun to implement.

Conceptually this involves:

  1. Comparing the root hashes of both trees.
  2. Continue recursion on the left and right children of the tree until the root hashes are equal.

The “Merkle root” stores the summary of all the transaction value in a singular value.

Simple Example

For example , if TA, TB,TC ,TD are transactions ( could be files, keys etc) and H is a Hash function. You can construct a tree by taking the transactions, hashing their concatenated values to generate children and finally reduced to a single root. In my scrawl above, this means hashing TA and TB, TC and TD, then hashing their concatenations H (AB), H(CD) to land at H(ABCD).Essentially keep hashing the until all the transactions meet at a single hash.

Example

Here’s an example that uses this technique to compare two files by generating their Merkle root to validate if they are equal of not (comments inline).

Invoke the script by calling “python merkle_sample.py “<file1>.csv” “<file2>.csv” to compare two merkle trees. Code below:

Key advantage here is that each branch of the tree can be checked independently without downloading the whole dataset to compare.

This translates to reducing the number of disk reads for synchronization though that efficiency needs to be balanced against the recalculation of the entire tree when nodes leave or go down. This is fundamental to Crypto currencies when transactions need to be validated by nodes and there is enormous time and space cost to validate every transaction which can be mitigated by Merkle trees in logarithmic time instead of linear time.  The Merkle root get put into the block header that gets hashed in the process of mining and comparisons are made via the Merkle root rather than submitting all the transactions over the network. Ethereum uses a more complex variant of the Merkle, namely the Merkle Patricia tree.

The applications of this range beyond blockchains to Torrents, Git, Certificates and more.