Archive+MD5 and other checksum ideas

Robert Connolly robert at linuxfromscratch.org
Sun Feb 17 15:20:51 PST 2008


On Sunday February 17 2008 04:39:16 pm Kevin Day wrote:
> This is sort of off topic and not directly related to HLFS, but
> considering the subject I thought I'd shoot this out at you guys.
>
> Transparently Transmitting Checksums With Archives
> I have recently had my mind on MD5 sums for file downloads and such.
> I put very little thought towards md5sums beyond normal use, but I
> started to think that this could be made easier.
> It seems to me that md5sums would be easier to transmit with the
> actual file and not separately.
> The idea is to make md5 checksums implicit.

I had/have a problem with my system freezing, and sometimes there's data loss. 
This is what backups are for.

So I made a bash alias to detect damaged tarballs:

# Using '/bin/tar' is important.
function tar() {
  if [ $1 == "xf" ] || [ $1 == "xvf" ] ; then
    if [ -f ${2} ] ; then
      if [ -f ${2}.sig ] ; then
        gpg --verify ${2}.sig &&
        /bin/tar $1 $2
      elif [ -f ${2}.sign ] ; then
        gpg --verify ${2}.sign &&
        /bin/tar $1 $2
      elif [ -f ${2}.asc ] ; then
        gpg --verify ${2}.asc &&
        /bin/tar $1 $2
      elif [ -f ${2}.md5 ] ; then
        md5sum --check ${2}.md5 &&
        /bin/tar $1 $2
      elif [ -f ${2}.sha1 ] ; then
        sha1sum --check ${2}.sha1 &&
        /bin/tar $1 $2
      elif [ -f ${2}.sha ] ; then
        sha1sum --check ${2}.sha &&
        /bin/tar $1 $2
      else
        /bin/tar $1 $2
      fi
    fi
  fi
}

There are bugs with it, but it generally works.

When I download anything I always look for signature files. All the files on 
gnu.org have them. It might be a good idea to make a file fetching script to 
attempt to download signature files with the archive.

> For a given compressions format, say gzip, first compress the
> particular archive and then make an md5 checksum for the particular
> archive.
> Once both of those are setup,wrap the original archive and its
> checksum in a second (tar) archive.
> Any application might be able to untar the original file and then run
> a checksum automatically and only continue if the checksum passes.
> This method would require no installation of anything new.
> Something could be installed to handle the extraction and auto-check
> the checksum on extract.

I would not want to create the md5sum myself unless it's a last resort. An 
externally available checksum lets me know that the download also went 
correctly. I also like to get the .md5 file from a different site than what 
the tarball came from.

Gpg of course does all this, with a database, and more. And tripwire if you 
want to do this with local files.

robert
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.linuxfromscratch.org/pipermail/hlfs-dev/attachments/20080217/6989fd79/attachment.sig>


More information about the hlfs-dev mailing list