Failed to Push Some Refs | Git Error
I was trying to push File size more than 100 MB to Github. Encountered The error was larger file detected, this exceeds Github’s file size limit of 100.00 MB. Let’s look into how to push large files to Github| Git lfs tutorial(Github Large File storage). Mostly when working with large Dataset this is a limitation.
Table Of Content
What is Git LFS?
LFS is Github’s Large File storage system, Where Github provides storage service to file greater than 100 MB, You might need this when working with a larger dataset. To get to know about the LFS storage space in your account you can go to settings in Github and click on Billings and Plans.
Github provides 1 GB of free file storage for everyone, any files beyond that the user needs to pay as they go. The below is the Dataplan if you need to purchase one. Now let’s upload the bigger file to the Github Repository. We will discuss how to upgrade Github Data plan.
How to install git lfs
Step 1: Go to the below link and download the file for whichever system you are using once, download the installation is pretty simple like any other software. The file size is 7.4 Mb for git lfs install.
Step 2: Open CMD(Command Prompt) and go to the directory where you want to perform this, it can be your cloned repository file or the new folder you have created.
If using windows system Type DIR to view to current directory Type cd foldername to move to the new exisiting directory
Step 3: Setting up Git LFS, Type the below code in the command promt to install and track the files in the folder.
In the second code what the system will do is track the CSV files in your local repository, In case your dataset is “mpg” then instead of CSV you can give mpg. Now let’s see how to push large file to Github.
Step 4: Now add the below code and add all the files your required, as my dataset was there in 3 excel in Data Folder, so I used the git add * command rather than adding the individual file.
Everytime when the people pull or push to the repository it will cound towards the data store bandwidth and utilize the storage left, if auto debit is enabled the money will deducted from the account.
The best approach is store the dataset in different folder and push the python notebook to Github. Use Kaggle storage to do in more efficent way.
FINAL VERDICT :
In conclusion, I hope you enjoyed reading this article on “Push large files to Github| Git lfs tutorial”. Let us know in case you find any issues in executing the above steps. Signing off Sanjay Kv
