Monday, March 13, 2006

Creating Your Own Debian Package Mirror for Use With Apt

Creating a Debian mirror is fairly easy using the debmirror command. In this article, I'll take you step-by-step through the process, including showing you how to configure the mirror server for use over FTP or HTTP/HTTPS. Note: The following was tested under Debian Sarge (stable).

Installation

First, we install debmirror and gnupg, if the latter is not already installed. When I initially tried to get this working, I ran into problems trying to use debmirror, without having imported the Debian master archive signing key - the quickest solution was to just import the public key into my GPG keyring. Note that this does not imply that you trust the key (in the GPG sense), it just imports it so the debmirror script will run.
apt-get install debmirror gnupg
Import the Debian master archive signing key:
gpg --recv-keys 2D230C5F
OR
wget http://ftp-master.debian.org/ziyi_key_2006.asc
then import this key into your keyring with
gpg --import ziyi_key_2006.asc

Building the Mirror

You'll need about 9GB of space for the full i386 sarge archive, all distributions (contrib, main, and non-free). Note that this does not include any source packages. Here is the command syntax: debmirror -v -a i386 -h ftp.us.debian.org -d sarge /path/to/mirror --nosource --progress Where /path/to/mirror is the path on your server where the mirror is going to be housed.

Archive Access Methods

Apache v1 Edit /etc/apache/httpd.conf or /etc/apache-ssl/httpd.conf:
Alias /debian /path/to/mirror

<location>
order deny,allow
deny from all
allow from all
Options Indexes FollowSymLinks MultiViews
</location>
Apache v2 Edit /etc/apache2/apache2.conf (Apache v2 does not have a separate SSL directory for config files, just an ssl.conf in /etc/apache2. See Setting up an SSL Server with Apache2 if you need it).
Alias /path/to/mirror "/debian/"

<directory>
AllowOverride FileInfo AuthConfig Limit
Options Indexes SymLinksIfOwnerMatch IncludesNoExec
</directory>

FTP Install vsftpd on the apt server:
apt-get install vsftpd
Change the home directory of the "ftp" user to /path/to/mirror using the vipw command.
ftp:x:108:65534::/path/to/mirror:/bin/false
The vsftp installation is automatically anonymous-ftp enabled on Debian, so you don't have to do anything else to get apt-get to work with this FTP setup. Then, put the following in your client's /etc/apt/sources.list: deb http(s)://debian/ sarge main contrib non-free OR deb ftp:// sarge main contrib non-free Then run apt-get update apt-get dist-upgrade You can re-run the above debmirror command from cron to automatically update the mirror however often you like. Make sure to use the Debian mirror list. Cron sample - this updates the mirror every morning at 2AM: 0 2 * * * debmirror -v -a i386 -h ftp.us.debian.org -e ftp --passive -d sarge /opt/debian --nosource --progress > /dev/null 2>&1 Technorati Tags: , ,

8 comments:

Anonymous said...

Hi Thanks for the great how to although i have one question it seems as though my Packages.gz are not inside the distribution and as such when my machine or another looks towards the mirror we get the error /pathtomirror/dists/sarge/main/binaray-i386/Packages.gz 404 not found

Must i create my own packages and if so how

Thanks in advance for any clue

Doug said...

Just guessing that perhaps you are not using the right line is sources.list? It should be something like:

deb http://debian/ sarge main contrib non-free

What does yours look like?

Doug

Anonymous said...

Hi Doug

Thanks for the quick reply

I have the following failure exactly

Err ftp: sarge/non-free Release
Temporary failure resolving 'dists'
Err http://debian sarge/main Packages
404 Not Found
Ign http://debian sarge/main Release
Err http://debian sarge/contrib Packages
404 Not Found
Ign http://debian sarge/contrib Release
Err http://debian sarge/non-free Packages
404 Not Found
Ign http://debian sarge/non-free Release
Failed to fetch http://debian/dists/sarge/main/binary-i386/Packages.gz 404 Not
Found




These are my 2 entries in the sources list

deb http://debian/ sarge main contrib non-free
deb ftp:// sarge main contrib non-free


I have used piwd to change the ftp user to where the mirror is store

/mnt/hda3

I also noticed it was needed to add debian to the hosts file afterlocal host to get the ftp slightly further

I have also tried 2 types of configuration for the apache2

Alias /mnt/hda3 "/debian/"
Directory #imagine tag here as not allowed post
AllowOverride FileInfo AuthConfig Limit
Options Indexes SymLinksIfOwnerMatch IncludesNoExec
Directory

and

Alias /debian/ "/mnt/hda3"
Directory "/mnt/hda3"
AllowOverride FileInfo AuthConfig Limit
Options Indexes SymLinksIfOwnerMatch IncludesNoExec
Directory


But it still has the same problem

Shane

Anonymous said...

Hi Doug

It is working from another machine with the ftp option.

The http does not work and the Packages.gz where in pathtomirror/.temp/dists/sarge/ and so on but i have made my own Package.gz files now and the contrib and non-free are working so far i am still waiting for the Packages.gz to be finished for main but i expect this to take some time longer due to the larger number of files.

Once again thanks for your help and the great how-to

Anonymous said...

Hi
plase help me
I Have debian-sarge 14CDs,and try to make
mirror-server
the configurations like this

#copy all CD one per one to direktory /debian/sarge
#cp-R /media/cdrom/* /root/debian/sarge
(what is true ?)

#dpkg-scanpackages pool/contrib /dev/null |gzip -9c
>Pakckages.gz
#dpkg-scanpackages pool/main /dev/null |gzip -9c
>Pakckages.gz
and
#cd /var/www
#lin -s /debian/sarge

apache:
/etc/apache2/sites-enabled/00-default
#RedirectMatch ^/$ /apache2-default/

for client
/etc/apt/sources
ded http://10.252.10.254 /sarge contrib main

test
#apt-get update
not success


Err http://10.252.10.254 /sarge/contrib Packages
404 Not Found
Ign http://10.252.10.254 /sarge/contrib Release
Err http://10.252.10.254 /sarge/main Packages
404 Not Found
Ign http://10.252.10.254 /sarge/main Release
Failed to fetch
http://10.252.10.254/dists//sarge/contrib/binary-i386/Packages.gz
404 Not Found
Failed to fetch
http://10.252.10.254/dists//sarge/main/binary-i386/Packages.gz
404 Not Found
Reading Package Lists... Done
W: Couldn't stat source package list
http://10.252.10.254 /sarge/contrib Packages
(/var/lib/apt/lists/10.252.10.254_dists__sarge_contrib_binary-i386_Packages)
- stat (2 No such file or directory)
W: Couldn't stat source package list
http://10.252.10.254 /sarge/main Packages
(/var/lib/apt/lists/10.252.10.254_dists__sarge_main_binary-i386_Packages)
- stat (2 No such file or directory)
W: You may want to run apt-get update to correct these
problems
E: Some index files failed to download, they have been
ignored, or old ones used instead.

Anonymous said...

Geek Pit

Doug said...

There is another way to do what you want - see http://www.debianplanet.org/node.php?id=698 for details, I think it will help.

Anonymous said...

Geek Pit
Help me
How to create mirror -debian-sarge
with source 14CDs
please step by step