How do you save all your installed pacages so you can easily reinstall them all on a fresh install of the OS?

I’m trying to make my system as crash-proof as I can so my question is to all of you gurus, how do you automate your package installing procedure from all the packages you know you will use/have?

pacman -Qqm lists foreign packages; which, for must users, means AUR  
pacman -Qqe lists packages that were explicitely installed. 

that said,

  pacman -Qqe | grep -v "$(pacman -Qqm)" > pacman.lst

and

  cat pacman.lst | xargs pacman -S --needed --noconfirm

These are the best backup/restore lines I've seen (don't remember where I got them tho).

that will install only the packages you don't have installed already as well as pull in any needed deps.  after that, -Syu and you're good to go.

I have found this and it seems like a decent alternative.

how do y’all do it?

1 Like

I just use a list but I look through it and select the packages I want on the new install separately as sometimes I have programs that i no longer want/require
pacman -Qe > list.txt

That being said I don’t install to many extra programs and normally just install what I need as needed

8 Likes

This suggests to not fall in the cliff. Not how to get out of it.

I would suggest looking for safe habits and precautions, so you won’t need to reinstall.

Frankly, I have never reinstalled Arch. I only fix it when it breaks. :person_shrugging:

1 Like

Same here.
I just backup my configs from one install and restore them in the other on the go when needed.

3 Likes

When I’m considering using a different distro, I do some trial runs in VirtualBox, and make a list of packages, tweaks etc. so if/when I install on my hardware it goes as smoothly as possible. I then update that list whenever I add or remove packages.

With the user_pkglist.txt function in :enos: this makes it very easy to have most of the apps/packages you want available on first boot after installation.

https://discovery.endeavouros.com/installation/customizing-the-endeavouros-install-process/2022/03/

5 Likes

Not so long ago I wrote a tiny script to facilitate moving settings and pkglists to other systems

#! /bin/bash
#
# Script for backing up configuration and package lists
#
#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the Affero GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see <https://www.gnu.org/licenses/>.
#
# @linux-aarhus - root.nix.dk
#
# Modify as necessary
filelist=('.bash_profile' '.bashrc' '.netrc' '.profile' '.zshrc')
# example folderlist=('.config' '.local' '.gnupg' '.mozilla' '.ssh' '.thunderbird')
folderlist=('.config' '.local')

archive_file="dotconf.tar.gz"
repo_pkg_file="repo-pkglist.txt"
cust_pkg_file="cust-pkglist.txt"
# /Modification

SCRIPTNAME=$(basename "$0")
VERSION="0.1"
if [[ -z $1 ]]; then
    echo ":: $SCRIPTNAME v$VERSION"
    echo "==> missing argument: PATH"
    echo "Usage:"
    echo "  $SCRIPTNAME /path/to/backup"
    echo "  Path to store output"
    echo "  e.g. $SCRIPTNAME /home/$USER/backup"
    echo ""
    exit
fi

set -e

if ! [[ -d $1 ]]; then
    mkdir -p $1
fi

conf_archive="$1/$archive_file"
repo_pkg_list="$1/$repo_pkg_file"
cust_pkg_list="$1/$cust_pkg_file"

# create an archive of common hidden files and folders

if [[ -e "$conf_archive" ]]; then
    # remove archive if exist
    rm -f "$conf_archive"
fi

todo=""
for file in ${filelist[@]}; do
    if [[ -f $file ]]; then
        todo+="${file} "
    fi
done

for folder in ${folderlist[@]}; do
    if [[ -d ${folder} ]]; then
        todo+="${folder} "
    fi
done

tar -zcvf "$conf_archive" $todo

# list packages from official repo
pacman -Qqen > "$repo_pkg_list"

# list foreign packages (custom e.g. AUR)
pacman -Qqem > "$cust_pkg_list"

echo " ==> Packagelists created"
echo "   --> $repo_pkg_list"
echo "   --> $cust_pkg_list"
echo " ==> Config archive created"
echo "   --> $conf_archive"
echo " ==> To install packages from lists"
echo "   --> sudo pacman -Syu --needed - < $repo_pkg_file"
echo " ==> To restore the configuration files run"
echo "   --> tar -zxcf $archive_file"
echo ""
9 Likes

My needs are relatively simple, it takes me 5 minutes to install the software I need. It may take me 30 to configure things (mostly web browser) to my satisfaction.
However, I usually just reinstall every now and then (6 months to one year) because I want to. Clear the cruft out (which I have installed something that I have very infrequent usage) and check out current defaults, not because I have to. The OS doesn’t rot (unlike that other prominent OS that seems to have bit rot built in as a feature).

3 Likes

This looks really interesting and looks to be what I’m looking for.

if I read this correctly without changing anything in the script it will store the output @ ~/backup? is this correct?

I agree with this, but I should have also mentioned it’s not just for disaster recovery, but an easy and fast way to be able to reinstall the distro or change distros for testing.

and being quite a noob with Linux, but want to learn, sometimes when following tutorials, that does not include everything or the documentation is not good like on secret-tool, stuff can brake, and I would not always be able to repair it. so I just want a good and straightforward way to get back to my working state if I had to reinstall the distro.

1 Like

mine too, but I would like to automate it as much as possible :slight_smile:

I’ve just tried your script and it works like a charm. thank you for letting me know about it.

During my first 6 months I broke stuff while figuring out how I wanted my machine to work and where I could get away with experimental software. During this I reinstalled a few times. Then later I reinstalled once more when I had got my machine mixed up too much with chaotic-aur.

Now I’m hoping to keep this install running for a long time.

When I was installing frequently I kept updating documentation on how to rebuild the system as I experimented with stuff. that documentation even includes my custom aliases.

1 Like

Question for you. when I try to run your script again, which worked first time. I now get this error.

tar: Cowardly refusing to create an empty archive
Try 'tar --help' or 'tar --usage' for more information. 

what am I missing, I was trying to run it again because I did not get two different files with pacman and aur packages.

The script is designed to run inside your home folder - because it requires write access.

It will generate 3 files

  1. a packagelist with packages from the repo
  2. a packagelist with cusom packages
  3. a tar archive with the files and folders specified in the script

To avoid errors when compressing - the set of files and folders are validated to exist and only added to the final todo-list if they do.

If you run outside you will have to supply the complete path to the files and folders you want to include otherwise the final todo-list will come up empty and thus making tar reply with the coward message.

Another option is to add a cd $HOME before the list validation.

The script could have been designed to imply the home dir but that would create an unnecessary complexity and would constrain the archive to be only home - as it is now it is more flexibile with respect to what you can include in the archive.

Hi @Root and thanks for the script!

I just tried it and while it was running I noticed that ~/.local/share/trash was also being backed up.
And mine was quite full!

Of course I should have payed more attention to the script and realized that the whole .local would be backed up.

Perhaps you could consider modifying the script to omit the trash folder?

Or you could create a specific list of folders to include from .local - omitting trash - or you could just kill the script - empty trash and rerun.

In fact the files and folders is merely suggestions - depending on your use case - you may even want to omit that folder and create your own detailed include lists.

The reason it is included in the example is because my usecase has a the bin folder, share/JetBrains folder and my Konsole custom profiles in .local/konsole thus creating archive of considerable size - we are talking several GB.

The script provided is not as detailed as my own - for brewity and to allow for customization to one’s need :slight_smile:

2 Likes

Right!
Thanks for the reply!

I use BTRFS snapshots & rsync to external media. If something goes wrong, I try snapshots and then rsync. I don’t mess up with package lists.

You should really take a look at Guix. Guix lets you save the entire configuration information from bootloader to installed packages etc. and reinstall while reading your preferences from config files.

That was it, I just had to move it to ~/ and now it works.