Knowledgebase:
A super elegant way to migrate user accounts from one Linux server to another
Posted by rbTech Staff, Last modified by rbTech Staff on 15 August 2014 09:56 AM
Thanks to nixcraft for his how-to here:

http://www.cyberciti.biz/faq/howto-move-migrate-user-accounts-old-to-new-server/.
I had written a very overcomplicated script to accomplish all this (never underestimate the power of AWK!!!), and this simplified my life immensely :)

Once I grokked all of the machinations below, I turned it into a slightly more elegant, less overcomplicated script that I've included as well.  Good luck, back up, don't blame me if you nuke your system, comments and suggestions to [email protected].  Flames to /dev/null :)

Setup UID filter limit:
# export UGIDLIMIT=500
Now copy /etc/passwd accounts to /root/passwd.mig using awk to filter out system account (i.e. only copy user accounts)
# awk -v LIMIT=$UGIDLIMIT -F: '($3>=LIMIT) && ($3!=65534)' /etc/passwd > /root/passwd.mig
Copy /etc/group 
# awk -v LIMIT=$UGIDLIMIT -F: '($3>=LIMIT) && ($3!=65534)' /etc/group > /root/group.mig
Copy /etc/shadow 
# awk -v LIMIT=$UGIDLIMIT -F: '($3>=LIMIT) && ($3!=65534) {print $1}' /etc/passwd | tee - |egrep -f - /etc/shadow > /root/shadow.mig

If you want the whole enchilada for moving users, here it is. Note: This is to be run AS ROOT from the NEW server (e.g. the server being migrated TO), and expects ssh key exchange to be done between the systems.

Be aware that you can well and hose your systems running things as root, and we hereby expressly disclaim any and all responsibility for any damage or problems of any sort caused by your use of this script. This code is supplied as a convenience and carries NO WARRANTY whatsoever.

#!/bin/bash

# Copyright (c)2010-2012 rbTechnologies, LLC
# By Rubin Bennett <[email protected]>

# Released under the terms and conditions of the GNU Public License version 2.


# A simple script to assist in server migrations from Linux to Linux
# Intended to be run on the NEW server, and expecting that you have performed
# ssh key exchange for password-less login to the OLD server.

# IP address or hostname of source server (e.g. server your're migrating
# AWAY from
sourceServer=172.16.1.55

function syncusers() {
echo -n "Do you have backups of your existing passwd files? [y|N] "
read
if [ "$REPLY" != "y" ]
then
        echo "Please back your files up and run this script again."
        exit 1
else
        scp $sourceServer:/etc/passwd /tmp/passwd.$sourceServer
        scp $sourceServer:/etc/group /tmp/group.$sourceServer
        scp $sourceServer:/etc/shadow /tmp/shadow.$sourceServer

        # First, make a list of non-system users that need to be moved.

        export UGIDLIMIT=500
        awk -v LIMIT=$UGIDLIMIT -F: '($3 >= LIMIT) && ($3 != 65534)' /tmp/passwd.$sourceServer > /tmp/passwd.mig
        awk -v LIMIT=$UGIDLIMIT -F: '($3 >= LIMIT) && ($3 != 65534)' /tmp/group.$sourceServer >/tmp/group.mig
        awk -v LIMIT=$UGIDLIMIT -F: '($3 >= LIMIT) && ($3 != 65534) { print $1 }' /tmp/passwd.$sourceServer \
| tee - |egrep -f - /tmp/shadow.$sourceServer > /tmp/shadow.mig

        # Now copy non-duplicate entries in to the new server files...
        while IFS=: read user pass uid gid full home shell
        do
                line="$user:$pass:$uid:$gid:$full:$home:$shell"
                exists=$(grep $user /etc/passwd)
                if [ -z "$exists" ]
                then
                        echo "Copying entry for user $user to new system"
                        echo $line >> /etc/passwd
                fi
        done </tmp/passwd.mig

        while IFS=: read group pass gid userlist
        do
                line="$group:$pass:$gid:$userlist"
                exists=$(grep $group /etc/group)
                if [ -z "$exists" ]
                then
                        echo "Copying entry for group $group to new system"
                        echo $line >> /etc/group
                fi
        done </tmp/group.mig

        while IFS=: read user pass lastchanged minimum maximum warn
        do
                line="$user:$pass:$lastchanged:$minimum:$maximum:$warn"
                exists=$(grep $user /etc/passwd)
                if [ -z "$exists" ]
                then
                        echo "Copying entry for user $user to new system"
                        echo $line >> /etc/shadow
                fi
        done </tmp/shadow.mig
       fi

}

echo "Copying user accounts and passwords from /etc/passwd"
syncusers
echo "If you use Samba, you may need to manually move the machines group with GID 200..."

exit 0

(9 vote(s))
Helpful
Not helpful

Comments (18)
Pierre Couderc
18 July 2012 08:52 AM
Bon boulot!
markit
02 September 2012 02:25 PM
Great script, I've modified (simplified) to just work locally and when I tested I noticed two bugs:
line="$user:$pass:$uid:$gid:$full:$shell"
should be instead:
line="$user:$pass:$uid:$gid:$full:$home:$shell"
(you forgot $home! I don't tell you how badly I was able to login in the new system, lol)
and also, when you migrate groups, you have:
echo $line >> /etc/passwd
that should be
echo $line >> /etc/group
instead.
Thanks a lot for the script
Matt W
05 December 2012 11:26 AM
Shouldn't this line include a closing back-tick:

exists=`$(grep $user /etc/passwd)
Rubin Bennett
14 December 2012 01:22 PM
Thanks for the comments and corrections, I've updated the scripts above to reflect the changes!
markit
26 January 2013 05:51 AM
In a comment to this article I've found a new good suggestion:
http://www.ghacks.net/2010/02/10/migrate-users-from-one-linux-machine-to-another/
that
awk -v LIMIT=$UGIDLIMIT -F: '($3 >= LIMIT) && ($3 != 65534) { print $1 }' /tmp/passwd.$sourceServer | tee - |egrep -f - /tmp/shadow.$sourceServer > /tmp/shadow.mig
should become:
awk -v LIMIT=$UGIDLIMIT -F: '($3 >= LIMIT) && ($3 != 65534) { print $1":" }' /tmp/passwd.$sourceServer | tee - |egrep -wf - /tmp/shadow.$sourceServer > /tmp/shadow.mig

otherwise the i.e. a "mo" user will catch also something like daemon and haldaemon
Rubin Bennett
16 July 2013 09:31 AM
Hi, markit
Your suggestion is good, though the UGIDLIMIT should prevent system accounts (those with UID/ GID under a sane limit, usually 500) from being brought over. I generally let the installers for the additional software create the user accounts on the new server, rather than bringing the accounts over as part of the user migration.
Thanks for your suggestions!
Rubin
Steve Louis
24 October 2013 04:47 PM
I am getting an error :
line 26: syntax error near unexpected token `|'
line 26: ` | tee - |egrep -f - /tmp/shadow.$sourceServer > /tmp/shadow.mig'

I have the key exchange done and have no issue going from new server to Old..
Your help would be appreciated!
Rubin Bennett
08 November 2013 02:27 PM
Hi, Steve, it looks like the line break was poorly executed by the TinyMCE editor we use to publish articles, you could just join that line with the one before and delete the backslash. The dangers of cutting and pasting (both when uploading and downloading scripts from the Internets!) :) Hope that helps!
Pankaj
30 December 2013 06:50 AM
Hi Guys
i want to copy users from one linux server (REL) to another (REL) and other server has already some users existing (same name as on source linux server) and i dont want to overwrite the existing users, can you please guide me how to achieve this.
Rubin Bennett
09 January 2014 09:39 PM
Hi, Pankaj
Actually the script already skips over users that exist on the new machine - the line:
if [ -z "$exists" ]
is the conditional that says only execute the following if the variable $exists is a zero value (e.g. the user doesn't exist in the new /etc/passwd).
Jeff Andersen
22 January 2014 12:12 PM
It appears that there's a bug on line 63: "exists=$(grep $user /etc/passwd)" should probably be "exists=$(grep $user /etc/shadow)"
Rubin Bennett
22 January 2014 06:18 PM
Hi, Jeff
It's a good catch, and one that I've fought with a few times in the past. There's a reason for the supposed bug though: the issue is that occasionally (and wrongly) there will be entries in /etc/shadow for non-existent users. Sometimes lazy sysadmins simply delete entries in /etc/passwd rather than using userdel, for example. So to be really correct, we should probably check both files before adding the entry to /etc/shadow. For the time being however, we check the users we import against /etc/passwd.
Joan
04 June 2014 09:44 AM
Hi Rubin, about the last comment, checking against passwd won't work because on the first loop the users will be already copied /etc/passwd, so we won't be copying the passwords never. The simplest solution would be to move the shadow loop to the top, so it wouldn't collide with the other.
Rafael Linux User
26 October 2015 04:18 AM
Long time of this thread, but I need to know something. My scenario is similar, but I have the old partition entirely backuped and I have now a new installation from scractch. The "root" password is the same than in my old system and I have access to that backup of the old machine.
Will this script work if I change "scp" and other commands to "cp" and so on?

Thank you
Rubin Bennett
16 November 2015 11:12 AM
@Rafael, I think the best way to do what you want would be to comment out the scp section, and place the new files where the scp commands would have put them. Then you would just run the remainder of the script.
Santosh Prasad
05 April 2016 05:47 AM
Why the fetching of /tmp/shadow.mig is diff from /tmp/passwd.mig or /tmp/grpup.mig
What is the specfic reason to use tee,{print $1} , egrep and - in finding shadow.img
pls help to to understand this.
Rubin Bennett
07 April 2016 10:53 AM
Hi Santosh
The script looks for users with UIDs greater than 500. There are no UIDs in /etc/shadow, only usernames, so the script loops through /etc/password, parses the usernames based on the requirements - UIDs greater than 500 and not 65534 (nobody), and grabs the line containing the username from /etc/shadow.
Mario Lobo
27 April 2016 11:32 AM
Hi Rubin!

First, let me thank you for this script.

I modified it a little. First, I copy the original files to a temp location so I can safely modify them, and use the originals for comparison. That way, I will avoid checking a modified file, and keep the "different" uid of system users of the new system if they exist on both.

It worked for me because I'm migrating users from a 9.x ubuntu to a 16.04 ubuntu.

And I added provisions to migrate gshadow as well.

After the migration, you will only have to replicate manually user allowances to system groups.

I imported the files manually from the old system so I commented from thge script the relevant sections.

Here is the script:

#!/bin/bash

# Copyright (c)2010-2012 rbTechnologies, LLC
# By Rubin Bennett <[email protected]>

# Released under the terms and conditions of the GNU Public License version 2.


# A simple script to assist in server migrations from Linux to Linux
# Intended to be run on the NEW server, and expecting that you have performed
# ssh key exchange for password-less login to the OLD server.

# IP address or hostname of source server (e.g. server your're migrating
# AWAY from
sourceServer=192.168.0.1

function syncusers() {
echo -n "Do you have backups of your existing passwd files? [y|N] "
read
if [ "$REPLY" != "y" ]
then
echo "Please back your files up and run this script again."
exit 1
else
# scp $sourceServer:/etc/passwd /tmp/passwd.$sourceServer
# scp $sourceServer:/etc/group /tmp/group.$sourceServer
# scp $sourceServer:/etc/shadow /tmp/shadow.$sourceServer

# First, make a list of non-system users that need to be moved.

# export UGIDLIMIT=500
# awk -v LIMIT=$UGIDLIMIT -F: '($3 >= LIMIT) && ($3 != 65534)' /root/oldpass/passwd > /tmp/passwd.mig
# awk -v LIMIT=$UGIDLIMIT -F: '($3 >= LIMIT) && ($3 != 65534)' /root/oldpass/group >/tmp/group.mig
# awk -v LIMIT=$UGIDLIMIT -F: '($3 >= LIMIT) && ($3 != 65534) { print $1 }' /root/oldpass/passwd | tee - |egrep -f - /root/oldpass/$
# awk -v LIMIT=$UGIDLIMIT -F: '($3 >= LIMIT) && ($3 != 65534) { print $1 }' /root/oldpass/group | tee - |egrep -f - /root/oldpass/g$


cp /etc/passwd /tmp/passwd
cp /etc/group /tmp/group
cp /etc/shadow /tmp/shadow
cp /etc/gshadow /tmp/gshadow

# Now copy non-duplicate entries in to the new server files...
while IFS=: read user pass uid gid full home shell
do
line="$user:$pass:$uid:$gid:$full:$home:$shell"

# if doesn't exist in the original
# copy to new

exists=$(grep $user /etc/passwd)
if [ -z "$exists" ]
then
echo "Copying entry for user $user to passwd"
echo $line >> /tmp/passwd
fi
done </root/oldpass/passwd

while IFS=: read group pass gid userlist
do
line="$group:$pass:$gid:$userlist"
exists=$(grep $group /etc/group)
if [ -z "$exists" ]
then
echo "Copying entry for group $group to group"
echo $line >> /tmp/group
fi
done </root/oldpass/group

while IFS=: read user pass lastchanged minimum maximum warn dum1 dum2 dum3
do
line="$user:$pass:$lastchanged:$minimum:$maximum:$warn:$dum1:$dum2:$dum3"
exists=$(grep $user /etc/passwd)
if [ -z "$exists" ]
then
echo "Copying entry for user $user to shadow"
echo $line >> /tmp/shadow
fi
done </root/oldpass/shadow
fi

while IFS=: read group pass gid userlist
do
line="$group:$pass:$gid:$userlist"
exists=$(grep $group /etc/group)
if [ -z "$exists" ]
then
echo "Copying entry for group $group to gshadow"
echo $line >> /tmp/gshadow
fi
done </root/oldpass/gshadow

}

echo "Copying user accounts and passwords from /etc/passwd"
syncusers
echo "If you use Samba, you may need to manually move the machines group with GID 200..."

exit 0

Post a new comment
 
 
Full Name:
Email:
Comments:
CAPTCHA Verification 
 
Please enter the text you see in the image into the textbox below (we use this to prevent automated submissions).