Initial commit with the scripts to synchronize git with svn

This contains multiple scripts:
 - A script to synchronize an existing git repository with subversion
   via a git synch client
 - hooks for the git server to trigger the sync after a push
 - a hook for the client to reject every manual change
 - A script to create a new git server repository, along with the git
   synch client, starting from an existing subversion repository.
This commit is contained in:
Mario Fernandez 2012-05-30 11:58:42 +02:00
commit 4c12f4021f
7 changed files with 285 additions and 0 deletions

99
README.markdown Normal file
View file

@ -0,0 +1,99 @@
# _git_ SVN Sync
This repository is intended to provide synchronization between a
running SVN repository and _git_, so that we can get away from
subversion while the build jobs are ported.
## Workflow
The idea is to use pure _git_ exclusively. The subversion repository is
up to date and used to build artifacts and jars, but nobody is
expected to write to it except the _git_ sync client.
Therefore _git_ would be used both at client and server side. This is an
improvement over using git-svn because it allows to commit branches to
the server, and avoids rewriting history when commiting to svn, among
other things.
## Technical view
For every project in subversion, two _git_ repositories are created, the
server and the sync client.
### _git_ Server
It is a normal bare repository, which supports every git
operation. Every developer clones this repository and uses it
exclusively for the project.
When someone pushes changes to the master branch, a hook is run which
uses the sync client to bring the changes to subversion.
### _git_ Sync client
This is a repository which is a clone of the _git_ server. When the
post-receive hook at the server is activated, the following happens at
this client:
* Changes are pulled from the server to the master branch.
* The master branch is merged into a svn sync branch.
* The changes are sent to subversion via git-svn.
This repository is not intended for developers to use. It rejects
every push and commit, and should only automatically sync with the
server.
### Maintaining consistency
The _git_ server and subversion should be in the same state at every
time. To guarantee this, the following conditions are required:
* Only the _git_ sync client should ever send changes to
subversion. The write access to svn should be restricted to the
remote _git_ user.
* Nobody except the build jobs in jenkins uses subversion directly
anymore. Developers interact only with the _git_ server.
* The _git_ sync client is never modified. It only pulls changes from
the _git_ server (only fast-forward allowed).
The consistency is assured via hooks that are installed at the server
and sync client. Access to subversion has to be configured separately.
### Reporting
If something does not work correctly, a mail will be sent specifying
the project which had the problem and the registered error
## Setup
### Initial setup
The machine where the repositories are installed needs the following
environment variables (defined in its ~/.bashrc):
* **GIT_SCRIPTS**: directory where the _git_ sync scripts are located
* **GIT_BASE**: directory where the _git_ repositories are stored.
* **GIT_SVN_SYNC_BASE**: directory where the sync repositories are stored.
* **GIT_SVN_SYNC_BRANCH**: name of the branch that is synchronized
with subversion.
This repository should be cloned in the directory **GIT_SCRIPTS**.
### SVN User
Git needs to have write access to subversion.
### Git config
For the git user that will sync with svn
git config --global user.email "the@email"
git config --global user.name "Git User"
### New project
Each project in subversion can be initialized with the
install/git-repository-from-svn.sh script. It makes sure that the
initial setup is carried and that the hooks are activated.

61
git-sync-with-svn.sh Executable file
View file

@ -0,0 +1,61 @@
# -*- mode: Shell-script-*-
#!/usr/bin/bash
#
# Author: Mario Fernandez
#
# Syncs git repository with subversion, using an extra git client.
#
# The client is a clone of the git repo. It has two branches:
# - master: It is sync'ed with the git repo. Should always
# fast-forward.
# - GIT_SVN_SYNC_BRANCH: Sync'ed with SVN (via git-svn). Noone else
# can write to svn.
#
# The changes from the git repo are pulled into master, and then
# merged to the svn sync branch. This branch is then synchronized with
# subversion.
#
# Required environment variabless:
# - GIT_SCRIPTS: directory where the git sync scripts are located
# - GIT_SVN_SYNC_BASE: directory where the sync repositories are
# stored.
# - GIT_SVN_SYNC_BRANCH: name of the branch that is synchronized with
# subversion.
#
# Usage: git-sync-with-svn.sh project_name
destination=receiver@host.com
project=${1?No project provided}
location=${GIT_SVN_SYNC_BASE}/${project}
if [ ! -d $location ] ; then
echo "The folder where the synchronization repository is supposed to be does not exist"
exit 1
fi
unset GIT_DIR
cd $location
report () {
echo $1
sh ${GIT_SCRIPTS}/report-error.sh $destination "$project" "$1"
}
# Get changes from git repository
echo "Getting changes from git repository"
git checkout master || { report "Could not switch to master" ; exit 1; }
if [ -n "$(git status --porcelain)" ] ; then
echo "Workspace is dirty. Clean it up (i.e with git reset --hard HEAD) before continuing"
exit 1
fi
git pull --ff-only origin master || { report "Could not pull changes from git repository" ; exit 1; }
# Synchronize with SVN
echo "Synchronizing with SVN"
git checkout ${GIT_SVN_SYNC_BRANCH} || { report "Could not switch to sync branch" ; exit 1; }
# In case of conflicts, take the master, as we are sure that this is
# the correct branch
git merge -Xtheirs master || { report "Could not merge changes into sync branch" ; exit 1; }
git svn dcommit || { report "Could not send changes to svn repository" ; exit 1; }

View file

@ -0,0 +1,58 @@
# -*- mode: Shell-script-*-
#!/usr/bin/bash
#
# Author: Mario Fernandez
#
# Initializes a git repository that is synchronized with an existing
# svn repository.
#
# Required environment variabless:
# - GIT_SCRIPTS: directory where the git sync scripts are located
# - GIT_BASE: directory where the git repositories are
# stored.
# - GIT_SVN_SYNC_BASE: directory where the sync repositories are
# stored.
# - GIT_SVN_SYNC_BRANCH: name of the branch that is synchronized with
# subversion.
#
# Usage: git-repository-from-svn.sh project svn_url
if [ -z "${GIT_SCRIPTS}" ] || [ -z "${GIT_BASE}" ] || [ -z "${GIT_SVN_SYNC_BASE}" ] || [ -z "${GIT_SVN_SYNC_BRANCH}" ] ; then
echo "The following variables are required for the synchronization to work: GIT_SCRIPTS GIT_SVN_SYNC_BASE GIT_SVN_SYNC_BRANCH"
exit 1
fi
project=${1?No project name provided}
svn_url=${2?No svn url provided}
location=${GIT_BASE}/${project}.git
client=${GIT_SVN_SYNC_BASE}/${project}
if [ -d $location ] ; then
echo "The folder for the git server already exists"
exit 1
fi
if [ -d $client ] ; then
echo "The folder for the git sync client already exists"
exit 1
fi
# Git Server
git init --bare ${location} || { echo "Could not initialize git server at ${location}" ; exit 1; }
# Sync client
git svn clone ${svn_url} ${client} || { echo "Could not clone svn repository at ${svn_url} in ${client}" ; exit 1; }
cd ${client}
git remote add origin ${location} || { echo "Could not set up server as remote from sync" ; exit 1; }
git push origin master || { echo "Could not sync client with server" ; exit 1; }
git branch ${GIT_SVN_SYNC_BRANCH} || { echo "Could not create svn sync branch" ; exit 1; }
# Set up hooks
for hook in pre-receive post-receive ; do
ln -s ${GIT_SCRIPTS}/server-hooks/${hook} ${location}/hooks
done
for hook in pre-receive pre-commit ; do
ln -s ${GIT_SCRIPTS}/sync-client-hooks/always-reject ${client}/.git/hooks/${hook}
done

19
report-error.sh Executable file
View file

@ -0,0 +1,19 @@
# -*- mode: Shell-script-*-
#!/usr/bin/bash
#
# Author: Mario Fernandez
#
# Sends an email with the error message obtained from syncing a repository.
destination=${1?No destination provided}
project=${2?No project provided}
message=${3?No message provided}
cat > /tmp/git-sync-failure <<EOF
The project $project could not be correctly synchronized. The output of the script was:
$message
EOF
subject="Git-SVN failed for project $project"
mail -s "$subject" $destination < /tmp/git-sync-failure

19
server-hooks/post-receive Executable file
View file

@ -0,0 +1,19 @@
# -*- mode: Shell-script-*-
#!/usr/bin/bash
#
# This hook is intended to be installed for a git server. It calls a
# script that synchronizes every change with subversion.
#
# Required environment variables:
# - GIT_SCRIPTS: directory where the git sync scripts are located
#
# Author: Mario Fernandez
master="refs/heads/master"
while read oldrev newrev refname
do
if [ "$master" == "$refname" ] ; then
sh ${GIT_SCRIPTS}/git-sync-with-svn.sh $(basename $PWD .git)
fi
done

23
server-hooks/pre-receive Executable file
View file

@ -0,0 +1,23 @@
# -*- mode: Shell-script-*-
#!/usr/bin/bash
#
# This hook is intended to be installed for a git server. It calls a
# script that synchronizes every change with subversion.
#
# Required environment variables:
# - GIT_SCRIPTS: directory where the git sync scripts are located
#
# Author: Mario Fernandez
if [ -z "${GIT_SCRIPTS}" ] || [ -z "${GIT_SVN_SYNC_BASE}" ] || [ -z "${GIT_SVN_SYNC_BRANCH}" ] ; then
echo "The following variables are required for the synchronization to work: GIT_SCRIPTS GIT_SVN_SYNC_BASE GIT_SVN_SYNC_BRANCH"
exit 1
fi
while read oldrev newrev refname
do
if [ "$refname" == "${GIT_SVN_SYNC_BRANCH}" ] ; then
echo "It is not allowed to push a branch named ${GIT_SVN_SYNC_BRANCH} to avoid conflicts when syncing with subversion"
exit 1
fi
done

View file

@ -0,0 +1,6 @@
# -*- mode: Shell-script-*-
#!/usr/bin/bash
#
# Author: Mario Fernandez
echo "The synchronization client does not accept manual modifications"
exit 1