LowEndBox - Cheap VPS, Hosting and Dedicated Server Deals

Dealing with Large Files on GitHub

Git LFSIf you’re using git for source code management, you may run into a problem if you try to include large files using GitHub or other repos.

By large I mean in the tens of megabytes range, and certainly larger files like ISO images, etc.

I ran into this for a project I’ve been tinkering with (full post coming soon) which generates large 6000×9000 .PNG images which come in at 30+ MB each.  The code is Python but I wanted to include some examples and when I tried to push to GitHub, git spun and spun and spun.  An hour later I killed it.

Turns out that git really doesn’t like large files.  Even worse, if you have large files that frequently change, your repo is going to be massive because all previous versions are changed.  So if part of your process is “rebuild the ISO and then update the repo” you are in for a world of pain down the road.

In my case, I just wanted a few .PNGs which are probably never going to change.  Best practice would be to store them separately outside the repo, but since I’m using GitHub it just makes sense to store everything in.

The solution to overcome GitHub’s limitation: git lfs.  From their GitHub:

Git LFS is a command line extension and specification for managing large files with Git.

The client is written in Go, with pre-compiled binaries available for Mac, Windows, Linux, and FreeBSD.

How to Use It

Installation is as simple as using your package manager.  On my mac I use Homebrew:

brew install git-lfs

Then set it up:

git lfs install

Now in your repo, you need to tell Git LFS what it should handle.  In my case, I said “all PNG files:

git lfs track "*.png"

Then you need to add the .gitattributes file where Git LFS stores its info:

git add .gitattributes

And then the rest is familiar:

git add example.png
git commit -m "added example.png"
git push -u origin main

After that, my giant .PNGs worked perfectly

 

raindog308

1 Comment

  1. Thank you for sharing! That is what I am looking for because I want to add large files on github.

    May 19, 2023 @ 12:06 am | Reply

Leave a Reply

Some notes on commenting on LowEndBox:

  • Do not use LowEndBox for support issues. Go to your hosting provider and issue a ticket there. Coming here saying "my VPS is down, what do I do?!" will only have your comments removed.
  • Akismet is used for spam detection. Some comments may be held temporarily for manual approval.
  • Use <pre>...</pre> to quote the output from your terminal/console, or consider using a pastebin service.

Your email address will not be published. Required fields are marked *