That’s the painful amount of time our developers spent screen watching or playing Pokemon Go while the code deployed to around 15 servers.
On a bit of probing, I realized; that the single-most time consuming activity which took 30 of the 40 minutes was syncing files to Amazon S3.
We were using s3cmd, a free command line tool for retrieving, uploading and managing data in Amazon S3. It is a very nice tool to sync files to S3, however, it is very slow.
So I decided to try some alternate tool which could reduce the time required for the sync. While searching for “alternatives for s3cmd”, my mediocre Googling skills introduced me to s3-cli.
As mentioned in its README, s3-cli is inspired by s3cmd and attempts to be a drop-in replacement. It is completely compatible with the existing s3cmd configuration file and supports most of the s3cmd commands. The most important advantage is that, while syncing directories, unlike s3cmd which uploads one file at a time, s3-cli uploads files in parallel thus making this activity really fast!
I installed it using npm on my server and the only change to the code was to replace the command “s3cmd” with the command “s3-cli”.
I ran the deployment script again and lo and behold! The result of making this small change was that our deployment time was reduced to