Day 8-10 - Data Science App in 45 days - minifying Docker, new progress, and new PM

Day 8-10 - Data Science App in 45 days - minifying Docker, new progress, and new PM

Tags
Engineering
Product
Published
Published July 27, 2022
Author
Hey guys, hope you’re doing great. We’re still sparing our free time despite of full time jobs working on this awesome project.
Although TagHub is by no means near finish, we’re received a lot of tractions from the community. Some students/programmers like to join the project as an intern, though I know the inner side team is messily organized and needed to be restructured heavily. We think we’re starting good.

Minifying dockers

notion image
After we decided that we will use docker as main deployment environment (in Heroku specifically), we started to get deeper into thinking of resizing the image size, because, by the time the image was huge. 700mb, f***, can you believe it? For a dead simple image with few packages..
As long as we dig deeper into this, we found that minifying docker can be done in two ways:
  • Using multi-stage build: use a surrogate environment (say “python3.8”) to install Python package then copying all compiled package to a new one with lighter image (say “python3.8-alpine”). See example
  • Using docker.slim: leverage deep analysis using docker.slim’s xray to analyze and remove unnecessary from the image
After using a combination of these two methods, we were able to cut down from 700mb (original) to 70mb (using multi-stage build) and then to only 20mb (after processed the image with docker.slim).
The reduction is huge! We can then save some space for the most important components on the web services - ML models.

Progress

At frontend side we’ve seen much progress from the developing the website on React. Since the last time we have the outline finished on Figma, Bach was working intensively on implementing this
We did have small mismatching between APIs and frontend requirements. Realizing and designing frontend/backend application is a lot harder than simple MVC model, it’s ok for us to take a step back to see the bigger picture.
We noted down the requirements from the user interface, and specs it so that backend apis can fulfill all the frontend features. We’ll be sharing the frontend design with you when it’s half done.. keep the best for last ya know.
On the backend, we refined the docs to look a little cleaner, splitting all apis to their proprietary sections.
notion image

New PM & a returned ML Engineer

This week we’ve seen a come-back from anh Thinh - a senior ML Engineer who works for a big corp. We’re working and designing and implementing upcoming APIs. Having a peer can really help alleviate the pressure you know :D
Also, we’re happy to share that Bao Nguyen - PM of Zalo/Momo Ex’s Product Associate is now joining the team. With his seniority in project management, I hope to see his engagement and impact he has to deliver to our team.
Until next time..