I recently moved some APIs for my side projects from AWS EC2 to a Hetzner self-hosted VPS. It's a great alternative because you get about 90% of the performance with 10% of the prices of AWS. If you know your way around Linux then self-hosting is the way (and not vercel).

## Creating an account
You can sign up for an account with Hetzner on their [website](https://accounts.hetzner.com/signUp). From there, fill up the relevant info and move to the payment method form. If you are in India, the page might only show a Credit card as the option but you can use a debit card as well if it has international transactions enabled. Fill in the card info and after a couple more steps your account will be created.

## Verfication
Hetzner requires you to verify your identity before you can use their services. To do this from India, you can use any of Aadhar, PAN, or Passport. If using Aadhar, enter the Aadhar number and wait for verification. More than likely you'll be met with a screen saying manual verification is required. At this screen you can take a picture of your face with the webcam and then scan the front and back of your Aadhar, aligning it within the frame. Submit these and the manual verification should be completed in ~5 minutes.

## Provisioning a VPS
After the verification process is completed, you can create a project from the Hetzner console. Then add a server to this project. Choose a region, OS image, etc. Choose vCPU or a dedicated CPU depending on your workload. Then check the IPv4 box if you need a public IPv4. It will increase the cost slightly (still much cheaper than EC2 public IPv4).

If you want public key auth for ssh then add your local public key in the next step. Otherwise, they will also email you the root password for ssh. The rest of the settings can be left as is for now. Click on Create Server.

## SSHing
If you used public key auth then you can ssh into your box using `ssh root@<your ip>`. Otherwise, use the ssh password you received in your mail.

How to purchase hetzner servers from India

## Getting Llama
Llama is being distributed on torrents for some time now. The legality is grey so please excersise judgement. The model can be obtained via [this](https://github.com/shawwn/llama-dl) GitHub repository. Follow the instructions and download the base models. The download size is around 220GB.

## llama.cpp
From their [GitHub](https://github.com/ggerganov/llama.cpp)
> Inference of Facebook's LLaMA model in pure C/C++

This project is a C++ implemetation and is a Apple silicon first-class citizen. It uses 4-bit quantization to run the model on a macbook.

This is a new project and is being developed rapidly. I'll update this post if any breaking changes happen.

## Setting up
Clone the [llama.cpp GitHub repository](https://github.com/ggerganov/llama.cpp). Now open this directory in a terminal and run `make` (must have gcc installed). This will compile the source files and result in 2 executables, `main` and `quantize`. We'll leave them for now.

Next up download the llama models and place them inside the `models` directory in `llama.cpp`. The `llama.cpp/models` directory should look like this now.

<picture class="" id="" ><source class="" id="" media="(max-width: 799px)" srcset="https://wjjzukdiudabpexheuxr.supabase.in/storage/v1/object/public/blog-assets/public/Screenshot2023-03-14at10503PM/Screenshot2023-03-14at10503PM-400.webp"/><source class="" id="" media="(min-width: 800px)" srcset="https://wjjzukdiudabpexheuxr.supabase.in/storage/v1/object/public/blog-assets/public/Screenshot2023-03-14at10503PM/Screenshot2023-03-14at10503PM-700.webp"/><img class="w-full" id="" src="https://wjjzukdiudabpexheuxr.supabase.in/storage/v1/object/public/blog-assets/public/Screenshot2023-03-14at10503PM/Screenshot2023-03-14at10503PM-400.webp" width="400" height="225" loading="lazy"/></picture>

Now we need to transform the `.pth` models from the downlaod to valid `ggml` files. For this we can use the `convert-pth-to-ggml.py` file. Make sure you have python3 installed using

```bash
python3 --version
```

Now install a few dependencies with `pip` or `conda`

For `pip` use:
```bash
pip3 install numpy torch sentencepiece 
```

For `conda` use:
```bash
conda install numpy torch sentencepiece
```

The llama models has 4 variants of different parameter size. You can choose which one to run based on your RAM size.

```
| RAM | Model         |
|-----|---------------|
| 16G | 7B and 13B    |
| 32G | 30B and lower |
| 64G | 65B and lower |
```

Based on the above table run the following to convert the models to `ggml`:
```bash
python3 convert-pth-to-ggml.py ./models/13B 1
```

This will take a while (~10-15 mins) depending on the model you selected.

Make the `quantize.sh` executable with
```bash
chmod +x quantize.sh
```
After the `ggml` files are generated, run the following command to quantize the models. The argument to quantize script would be the model you transformed to `ggml` in the previous step.
```bash
./quantize.sh 13B
```

This also might take around 10-15 minutes to complete.

When the quantization is complete you can run the inference by providing it a prompt. To do so enter the following command.
```bash
./main -m ./models/13B/ggml-model-q4_0.bin -p "Building a website can be done in 10 simple steps:" -t 8 -n 512
```

This command will load the model and genrate a response based on your prompt. If you get a segmentation fault or a very slow response here then that is probably due to low RAM.

The project is being worked on currently and will have faster load times for models soon. I'll update this post in case of a major change.

run inference on llama models on macbook m1

## table of contents

- [introduction](#introduction)
- [why optimize](#why-optimize)
- [about sharp](#about-sharp)
- [setting up a project](#setting-up-a-project)
- [specifying image sizes](#specifying-image-sizes)
- [aspect ratios](#aspect-ratios)
- [web friendly image formats](#web-friendly-image-formats)
- [finishing up the code](#finishing-up-the-code)
- [extra credits](#extra-credits)

## introduction

When dealing with user-submitted images it is important to optimize them so that the appropriate image can be loaded on different device sizes. In this article, I demonstrate how to build a simple optimization pipeline with [sharp](https://www.npmjs.com/package/sharp).

## why optimize

Optimization of images is important because more often than not the main culprits of a slow web app are poorly optimized images. Not only do improperly sized images contribute to most of the [cumulative layout shift (CLS)](https://web.dev/cls/), they also are a bottleneck for loading time on slower connections.

## about sharp

Sharp is an npm module that can be used for many image processing tasks. It is a low-level package, partially written in C++ on top of a library called [libvips](https://github.com/libvips/libvips), this makes it extremely fast and efficient even when editing large images with multiple transformation steps.

## setting up a project

**in a new folder, initialize an npm project using**

```bash
npm init -y
```
**now install the sharp module with**

```js
npm i sharp
```
Create a new folder src and add index.js file inside it, then create a folder named images and another named output.

> for ease of use you can add the following script to your package.json
```js
"scripts": {
  "dev": "node src/index.js"
}
```

Now add the [this 2.4MB image](https://unsplash.com/photos/ynBpPyCKRtw/download?force=true) to the images folder. 

**now let us import the required modules and read the image file inside src/index.js**

```js
import sharp from "sharp";
import fs from "fs";
import path from "path";

const fileName = "kier-in-sight-ynBpPyCKRtw-unsplash.jpg";
const imageDir = path.join(process.cwd(), "images");
const outputDir = path.join(process.cwd(), "output");

// this is just for demonstration so I am using 
// synchrounous method. In production you should 
// almost always use async version i.e. readFile

const imageBuffer = fs.readFileSync(imageDir + fileName);


```
Sharp takes many types of inputs. It can read an image in form of a string, a buffer, an IntArray and a few other [types](https://github.com/DefinitelyTyped/DefinitelyTyped/blob/2a34952b37fa54fdd47502b70a27ccff665ea1f2/types/sharp/index.d.ts#L26).

## specifying image sizes

The image sizes you generate will depend on what sizes you want to support or how precise you want to be with your sizes. I usually generate 4 standard sizes based on common CSS breakpoint lengths i.e. 320px, 768px, 1024px and 1280px. Of course, one can generate many more, this serves my case well enough and is a viable option for the free cloud storage I am using.

I usually add another dimension of 10px that produces pixelated image of a couple of hundred bytes. This helps with inlining a pixelated version of the image and lazy loading the original image. I will write a separate blog detailing how I use this approach for a smoother UX on the frontend but for now, we'll just add 10 to the list of widths.

**add the following line to src/index.js**
```js
const imageWidths = [320, 768, 1024, 1280, 10];
```

## aspect ratios

We have the image widths and can resize the image right away however sometimes you might want to use set a custom width and height ratio for the sake of consistency across images. For eg. a popular aspect ratio to use is 4:3. I use 16:9 on this blog. To force a 4:3 ratio, given the width, we can use some maths to get a height and pass that height to sharp.

```js
// calculate height when width is given to get a ratio of 4:3
function calc4_3Height(width){
  return Math.floor((width * 3) / 4);
}
```
> @warn now an obvious question is that wouldn't this destroy the natural look of the image?

and the answer is yes, but we can use an option built into sharp called `contain` to preserve the natural aspect ratio while producing an image of our desired ratio.

> **from the sharp [docs](https://sharp.pixelplumbing.com/api-resize)**

`contain`: Preserving aspect ratio, contain within both provided dimensions using "letterboxing" where necessary.

Here letterboxing means padding the parts of the image which can't be occupied by the original image. The color of the padding can be specified and I *mostly* use transparent.

## web friendly image formats

Choosing the correct file format is a crucial part of the optimization process. We ideally would like to have good compression along with the support of high color depth. I use [webp](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Image_types#webp_image) for all the images on my blog because of its exceptional compression. You can choose from any of the [common types](https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Image_types) of image formats, all with their tradeoffs.

> @err **modern formats like *webp* and *avif*, although great, are not supported by IE so choose wisely if you plan to support IE**

## finishing up the code

Keeping the above information in mind let us complete the resizing code

```js
// get the fileName without the extension
let fileNameWithoutExt = fileName.split(".").slice(0, -1).join("");

const sharpImage = sharp(imageBuffer);

// the sharp library uses promises and thus we can loop over all the
// imageWidths and store the promise returned by sharp in each iteration
// in an array (await can also be used with a for loop inside an async function)
const promises = [];

imageWidths.forEach(width => {

  const height = calc4_3Height(width);
  
  // note that sharp has many more options available but I am using only
  // those which I need to get the job done
  const promise = 
       sharpImage
       //pass and width to the resize function
      .resize(width, height, {
      
        // contain for preserving the aspect ratio inside given width/height
        fit: "contain",
        
        // a transparent background using the alpha value of 0
        background: { r: 255, g: 255, b: 255, alpha: 0 },
      })
      .webp() // change this with your required file type
      
      // write the output of above transformations to a file
      // I use naming convention of *filename-widthxheight*
      .toFile(`${outputDir + fileNameWithoutExt}-${width}x${height}.webp`);
      
  // at last push the promise returned by sharp to our array    
  promises.push(promise);
  
});

// here we could either use an async IIFE or a top level await if
// you have that configured. I'll just use and IIFE

(async () => {
  await Promise.all(promises);
  console.log("done resizing");
})();

```
That is it. You now have a process of resizing user submitted images to any width you want.

## extra credits

As you can see we are saving the files on the local disk which isn't always what you'd want. Ideally, we would upload the file onto cloud storage like S3. I use [supabase](https://supabase.io/) storage for saving my image files because it's free and fairly straightforward to use. You can find the implementation of this blog's image upload [here](https://github.com/msx47/blog/blob/master/src/pages/api/upload/index.ts). It is a **vercel** serverless function running on the free tier and is super snappy even for large file  sizes.

This was just part-1 of a 2 part series I have planned on image optimization. The next one would deal with the frontend and I'll explain how I used the low-res image we created here in nextjs.

**thanks for reading :)**

optimize images in nodejs using sharp

## starting out

After months of delaying and finding excuses, I am committing to working on a new year resolution of writing more articles.

## some context

Sometime last year, I bought the domain, `mohits.dev`, without having an actual plan about what I was going to use it for (always works great).

Around October last year, I decided on and started building a blog from scratch. 

> @warnWhy didn't I use something like `hashnode` to get a blog service up and running?

Well, because I find the process of building something complex from the ground up very appealing. Also, third-party services wouldn't have all the customization I wanted.

Thus, starting fall last year I slowly built this website. During this time I have learned a lot of things that I most definitely wouldn't have if I had just used a blog service.

## plans

I plan to write at least one article per week. I have never really done any serious writing before, so this should be fun. The articles would mainly be about something I learned or something I know that I think can benefit others, etc.

While developing this blog, I wanted to reduce the image loading bottleneck, which led me to explore image processing and how to optimize them for loading. Thus, the first blog would be about how to create a small image processing/optimization pipeline for resizing images using the [sharp](https://www.npmjs.com/package/sharp) npm module.

thanks for reading :)

Hello, I'm Mohit