The war against cloud and amazon prime microservicies - TechDataTalk #016
Welcome to a new edition of TechDataTalk!
This week I am coming back with the normal newsletter format and also with fresh content and a bunch of ideas for the next newsletters
Before starting the newsletter, I want to say that probably in July/August I will do a rebranding of my personal brand, the newsletter will continue with the same format, but with a new design, content…
This week I am going to talk about Amazon Prime microservices blog post and if it is always necessary to use a microservice architecture hosted in AWS…
Source: https://microservices.io/patterns/microservices.html
If you don’t know what I am talking about, the past week more less Amazon Prime Engineering Team published this post
In this post they explain how they have moved from a serverless architecture to a monolithic architecture, which has cut down their costs by a 90% and also there has been improvements in speed, latency…
Specifically, they have moved a private tool for video/audio quality, which was architected using AWS Step Functions similar to AWS Lambda, thay also used tools like AWS S3
They did this due to speed, as they need to get this tool into production as fast as they could, so in a first step, serverless was the best option
The problem of this architecture is that you are not taking into account the size of the video(the storage cost), the latency to pass the data between the different microservicies, the data shuffling…
Javi Santana writes a very good thread about this
https://twitter.com/javisantana/status/1654930040008605696
At the end, serverless solutions are a good option to deploy fast to production, but normally they are not the best option to scale
However, there is a lot of people that after reading this post have started to criticize microservicies and started to say that monoliths are better
As always happen, there is no better solution, each situaiton, each company, each product has differente needs which are key to take this type of decisions
Eventhough Amazon Prime has changed this tool to a monolith, they do not have a monolithic architecture in their app, it would be impossible to scale and to mantain the infraestructure
There is other point, that I do not have talk a lot about and is the war against cloud that some people have
Normally, cloud is a very good option for most of the “serious” apps, ok, if you have a woocomerce with 1k products you do not need it
But if you want to create ETLs piplines where you are going to store terabytes of data you will need it, if you want that architecture to scale
Obvously you can do it i a VPS but I think that it is not the best option….
👨🎓 Paper Of The Week
Sparks of Artificial General Intelligence: Early experiments with GPT-4 => https://arxiv.org/pdf/2303.12712v5.pdf
In this paper published by Microsoft, they compare the “human” capabilities of LLMs like GPT-4 along with others like Google PaLM-2
They demostrate how GPT-4 can solve complex problems, medical tasks… as a human can do it or in a very similar way
This do not demostrate that this tools are close to be an AGI, eventhough we are closer than we think, for example, this tools do not have consciousness
Before creating an AGI they sould create a real Multimodal model, probably the closer to this will be PaLM-2 which was presenteted by Google at Google I/O
PALM-2 paper: https://ai.google/static/documents/palm2techreport.pdf
🧪 This is F*cking Science
Neuromorphic sensorimotor loop embodied by monolithically integrated=> https://www.science.org/doi/10.1126/science.ade0086
Researches have develop a new type of electronical skin which can be integrated into your body without event noticed
This “skin” can send information to your brain as a normal skin can do
It has been tested in rats, and now they are going to test it on humans, if all goes well this could be an incredible advanced
Imagine that one person gets burn in a fire, this “e-skin” could substitute the dead skin that fire caused
If you want to read a less technical post, you can read this nature’s post
https://www.nature.com/articles/d41586-023-01684-9
📚 Biblioteca
To sum up, I am going to list some posts that I really like and I think they are useful:
So, we have come up to the end of the newsletter, neext week I will come back with some new content, if you have liked this edition, please share it on twitter and if you want, you can give some feedback