Git

You might want to git pull all repos you cloned in a specific folder before starting to work on these projects.

This is a simple script in both python and bash.

#!/usr/bin/python3
import os,sys
import git
from termcolor import colored
from argparse import ArgumentParser

def main(argv):
  parser = ArgumentParser()
  parser.add_argument("-d", dest="basedir", help="base directory", default='./')
  args = parser.parse_args()

  for root, dirs, files in os.walk(args.basedir):
    for repo in dirs:
      if repo.endswith(".git"):
        pullresult = git.Repo(os.path.join(root, repo).replace(".git","")).remotes.origin.pull('master')
        print(colored(str(os.path.join(root, repo).replace(".git","")), "green") + "\n" + str(pullresult))

if __name__ == "__main__":
   main(sys.argv[1:])

#!/bin/bash
find . -name ".git" | awk 'BEGIN{FS=OFS="."}{NF--; print}' | xargs -I{} git -C {} pull

A simple script to locally clone a list of repos and bundle them. Will skip empty repos and bundle wikis as well. One list file in the same path is required, one repo per line in the format: https://git:MYGITTOKEN@git.server.address/path/repo.git

# Directory that hosts bundles
mkdir bundles

# Bundle all non-empty repos
while read line; do
  git clone --mirror $line folder
  cd folder
  filename=$( echo $line | awk -F/ '{print $NF}')
  # is repo empty?
  if $( test -n "$(git rev-list -n1 --all 2>&1 2> /dev/null)" ) ; then
    echo "Git repo has commits, bundling.."
    git bundle create ../bundles/$filename.bundle --all
  else
    echo "Git repo has no commits, skipping"
  fi
  cd ..
  rm -rf folder
done < list

# Bundle all non-empty wikis (we might as well have empty repos with wikis)
while read line; do
  wikiurl=$( echo $line | sed 's/\.git/\.wiki\.git/g' )
  git clone --mirror $wikiurl folder
  cd folder
  filename=$( echo $line | awk -F/ '{print $NF}')
  # is repo empty?
  if $( test -n "$(git rev-list -n1 --all 2>&1 2> /dev/null)" ) ; then
    echo "Git repo has commits, bundling.."
    git bundle create ../bundles/$filename.wiki.bundle --all
  else
    echo "Git repo has no commits, skipping"
  fi
  cd ..
  rm -rf folder
done < list

Let’s begin saying that this should never ever happen. It is really bad practice. It completely defeats the purpose of using a version control system but you know, it isn’t all puppy dogs and rainbows out there.

Sometimes it happens that a partner accesses an on-premise software of ours in order to update some configs on his own. And yes, it is done in production changing a huge config file, often during the night. For our convenience this file is versioned on our gitlab server. A merge request is submitted by devs and a CI/CD pipeline is in charge of calling an ansible script that triggers a git pull after testing.

But hey, the pipeline obviously fails if the local file has modifications. We don’t really want to use a git reset –hard and destroy all the work done in production by our partner, but we don’t want our pipelines to fail miserabily because of local changes.

So here is a script that does this nasty job:

#!/bin/bash
cd /etc/my-onpremise-software
/bin/git fetch --allif [[ $(git status --porcelain --untracked-files=no | wc -l) -gt 0 ]]; then
  /bin/git add configfile
  /bin/git commit -m "Automatic forced push"
  /bin/git push --force
fi

You might be wondering why we do a git push –force. We want the actual working copy stored in production to have precedence over developers modifications on config file; in this case devs are aware that their work might get lost, that’s why working on branches in order to ask a new MR is essential.

Just add this to your crontab. The frequency depends on how often the changes are applied in production. Since this is a relatively sporadic event in our case once a day at 6am was ok for us.

But please, remember: do this only if it is a matter of life or death. Extremis malis, extrema remedia.