Kicking Off Project X: Custom Telegram Bot API Setup and File Caching Pipeline
I figured it out. I will do what happens in one of my favorite movies, 'The Social Network': periodically write about my progress on a new project. And while in the movie Zuckerberg is documenting work on the rather non‑ethical FaceMash, I will be sharing updates on a much more ethical project of my own. Let us call it Project X.
Here we go.
I have built and launched a telegram-bot-api server that lets me bypass some of the limitations of the default api.telegram.org, especially the ones that really matter to me:
1. Maximum upload file size
2. Maximum number of webhook connections
3. Ability to use a local directory to access files
I wrote a Python script that exposes the bot files over the web (so I do not have to rewrite a lot of code) and caches them in S3 for further processing, so they do not take up space on the server.
While writing this post, I realized this project will probably generate a lot of open-source code on GitHub, like this script.
Here is a short breakdown of how it works.
On first request:
0. telegram-bot-api server saves the file to a local directory on the server
1. The script checks whether the local file exists
2. Returns it
3. Uploads it to S3
4. If the used space exceeds the configured threshold, it deletes the file from the server
On subsequent requests:
0. Checks if the local file exists (in case it has not been deleted yet)
1. If there is no local file, checks whether it is in S3
2. Returns it
Notes to self for later: telegram-bot-api server exposes server statistics, which the 'Petersburg olympiad kids' of course return in a custom format instead of a Prometheus Exporter format that would make it easy to build graphs and alerts. I will have to write a Python parser/converter to translate these metrics into the format I need.
More to explore
Startup Taxes Between Estonia and Portugal: A Quick Reality Check
As a tax resident of an EU country who files my own returns, today is my quarterly 'Tax Day'. On this day I set aside a few hours to file social security report…
Saylify Update: Fighting Perfectionism, Refactoring, and Finding the Right Focus
I have not written anything about Saylify for a long time, even though I planned to launch in January. Unfortunately, life likes to throw in challenges you can …
Human-Like Memory for LLMs
TL;DR I wrote a manifesto-style essay about a memory model for LLMs that is as close as possible to human memory and lets the system build a relationship histor…
When Companies Finally Say the Ugly Part Out Loud
Now we are finally fucking talking. Not all that crap like "internal policies", "no explanation needed", "just because".1Office are the first who wrote it plain…