[lfs-support] More control and package management using package users

Rob Taylor rtaylor777 at gmail.com
Mon Oct 7 21:27:45 PDT 2013


I noticed that the original more_control_helpers.tar.bz2 file mentioned in
the "HINT"
http://www.linuxfromscratch.org/hints/downloads/files/more_control_and_pkg_man.txt

is not found using the link in the "HINT". The URL should be
http://www.linuxfromscratch.org/hints/downloads/files/ATTACHMENTS/more_control_and_pkg_man/more_control_helpers.tar.bz2.
Just in case people can not find the original file in the future I also
link to a copy of it near the bottom of my page here:

https://www.javacrypt.com/lfs



Dan,

The build script itself already exists in the
"more_control_helpers.tar.bz2" which is as it was before I got to it. I
altered it a little but for the most part it is as I found it. Now that is
not to say that I can not explain most of what it does... I can, however I
don't want to take credit for its design or function.



re: "Where can I find the 'root_{pre,post}_commands'?  They do not appear
in 'build.conf' and don't seem to be defined in 'build.' You call them as
for .configure, make and install and make the logs, but I can't find the
commands themselves."



I noticed that when a function is missing from the build.conf file, the
step is just skipped. It appears that Matthias was experimenting with this
and didn't include it in the build.conf. I haven't tested it, but the -c
switch given to the su command is generally how you can execute a script or
command as a different user. So calling the script itself with a specific
option mentioned, and executing it as an option to the su command should
theoretically execute that phase of the script as the specified user (in
this case root). I expect the build script would pause at this point
expecting the user to enter the root password since I don't see how it is
getting passed.



re: "I'm gonna go back to your build notes because I don't want to ask a
dumb question about build directories.  However, since you use 'ln -s' are
the xxxbuild and yyysrc directories just "dummies" or do they actually
exist? If they exist, what is their purpose?"



The extracted archive is created with a folder that is taken from the
archives name and is extracted into the folder called xxxbuild. The yyysrc
link is to this extracted directory. In this way no matter what package is
extracted the path $HOME/xxxbuild/yyysrc is always to the source that was
extracted. Not my design but it seems to work. For example



libpipeline-1.2.4.tar.gz



Would be extracted to $HOME/xxxbuild/libpipeline-1.2.4 and the link
$HOME/xxxbuild/yyysrc points to it.



re: "Last thing so you can continue to vacation.  I was interested to see
how you handled all the tar.{xz,gz,bz.bz2} files.  PATTERNS seems to be the
key.  In build.conf "PATTERNS='4.3.2.tar". I don't understand this.  Would
you explain it?"



The text in the PATTERNS variable is how the build script identifies the
files you wish to extract as part of the build. You place the patterns in
the order desired with spaces separating the patterns. Generally there is
only one archive to extract and as long as the PATTERN you use appears
somewhere in the file's name the extraction is successful. I usually only
put the text "tar" in the PATTERN as it matches nearly all package archive
names and yet does not usually match the names of patches which you may
also have in the $HOME directory of the package user.



re: "That's why after I built my first {,B}LFS system, I generated my own
install.dir.lst."

Part of the changes I did make, almost near the beginning of my efforts on
this project, was to add a script to generate and maintain the list of
"INSTALL" directories. You can see more details of what I changed and added
in my developer notes:
https://www.javacrypt.com/lfs/dev_user_package_management.txt

If you look at the source code for the wrapper scripts that I have been
working on, you will see they seem rather involved. I noticed that there
was a great variety of ways that packages used the commands: install, rm,
ln, sln, unlink, mkdir, chmod, chown, chgrp etc.. I also noticed that they
did things that don't even seem to be documented, such as placing folder
and file references first and options last, or putting the -m option
between source and target file and directory options. Also did you know
that it is perfectly acceptable to have a line feed in the middle of a file
name? One of the packages test this and so my wrapper scripts had to become
more and more robust to deal with these oddities.

But on that note it is worth mentioning that the wrapper scripts are not
really required until the "install" step. If you suspect that a "make
check" or "make test" is failing due to the wrapper scripts you can simply
disable them by changing the path, for the "build check" step.

For example, the rebuild of coreutils to enable "acl" in the BLFS was
checking things that were not permitted by the wrapper scripts... so I
simply did this in the build.conf file for the check function:

check_commands()
{ :
  # for whatever reason, the wrapper scripts caused make test to fail so...
  WRAPPER_PATH="$PATH";# back up the PATH
  export PATH="/bin:/usr/bin:/sbin:/usr/sbin";
  make test
  export PATH="$WRAPPER_PATH";# restore the PATH
}

I am creating a "my_blfs_7_4_notes.txt" file, to carry on from the
similarly named LFS file,  but I am a long ways off from publishing it.

Take Care,

Robert Taylor






On Sun, Oct 6, 2013 at 7:51 AM, Dan McGhee <beesnees at grm.net> wrote:



    On 10/05/2013 11:11 PM, Rob Taylor wrote:

>     Hi Dan,

>     Thanks for the feedback. I'm just on vacation for a couple of weeks
to Hawaii, the big island. So I may not seem very responsive to emails for
a bit.

    Oh goodness! I don't want to impact your vacation, but I'll still
post--so that you can respond if you think this stuff is as much fun as
Hawaii.  :)  BTW. I retired three years ago, so I am on continuous vacation.
I love it.  Besides, in one of my previous lives I was an engineer, and
right now I'm trying to line up all my ducks before I start my build. So
there is plenty of time.  There are a few things I want to play with during
my Ch. 5 efforts (see below) and that will slow me down.

>     Feel free to change or use anything I have created. I didn't really
take over the hint, partially because I am not sure how much time I will
have to put into supporting it. But I did work on the wrapper scripts
because I tried out this user package management idea and I could see room
for improvement. Anything you or anyone else can add to improve this
project is welcome.

    Which goes to my first question in this round.  As a general statement,
I'm closely reading the build and build.conf scripts.  All of my questions
stem from that.



    Where can I find the 'root_{pre,post}_commands'?  They do not appear in
'build.conf' and don't seem to be defined in 'build.' You call them as for
.configure, make and install and make the logs, but I can't find the
commands themselves. Ubuntu uses gedit for it's text editor and, when you
print the file, it puts things in nice pretty colors depending on what the
statement is.  The last technicolor statement in the build script is "echo
-n "Preparing for install(root)...". The next statement calls
root_pre_install_commands, and it is in black as is the rest of the printed
script.  I'm thinking that it can't find the commands.



    I downloaded the tarball yesterday.

>     ...Su no longer exists in coreutils so copying it in section 5 can
not occur. I see from the other effort mentioned here
https://github.com/ericherman/package-users that he suggests installing
shadow into the /tools area instead of /usr. That would make sense to avoid
any collision of files later when the normal shadow install occurs.

    This is one of the things I thought I'd play with after I learned about
no 'su.'  My building skills are a little stale, but they're slowly coming
back.  I seem to remember that you can 'make' or 'make install' individual
targets.  I'm wondering if you can do that in 'shadow' during Ch. 5.  I
think--remember I'm stale. please don't laugh--' make target=su' or 'make
install -su'. If it's possible, the syntax escapes me right now.  I seem to
remember that there are some packages in LFS that build only one or a few
targets in a package.  I'll review the syntax and see what happens.

>     I thought about changing my_lfs_7_4_notes.txt to use that approach
but then decided that the example of how to deal with file overwrite issues
is a useful demonstration of the scripts handling of these issues.

    Gonna re-read those to see if it helps with the 'root_*_install
commands' stuff.

>     6. The build script will still quit after a section such as the
"check" fails. What I was referring to is that rather than a command such as

>     install -c -m04755 -o root -g root ../../sourcefilename /sbin

>

>     failing and causing the entire "make install" to fail because the
package user does not have permissions to run it,

>     my wrapper scripts will filter out the -o root -g root and will alter
the -m04755 to be -m755 and will create a log

>     of the command as it was before and after modification, present
working directory when the command ran, any errors returned from

>

>     the command if it was run and in doing so the "make install" will
continue to run through to completion installing any and all

>     files that would have been run after the failure. So when you are all
done with a successful install, all you need to do is

>

>     examine the logs to see the list of files that you may or may not
want to make SetUID or SetGID root etc..

>

>     My wrapper scripts will also examine any issues of ownership of
files, log those issues and skip the specific command that would

>

>     have generated a permission error so that the "make install" will
continue as far as possible generating a complete list of these

>     files with ownership issues so that a single chown command can be run
to fix all these permissions issues before rerunning

>

>     "make install" again without issue. You really would have to
experience the process to appreciate what I mean. Also, suppose you don't

>     want this package to have replaced those specific files where there
was an overwrite situation? In this case if the overwrites

>

>     (which are logged for review) are the only issue with an install and
the install completed with a result of success, you can simply move on.

>     This is much more seamless than having to edit "Makefile.in" files
removing or fixing offending entries and running

>

>     "make install" over and over again until it finally succeeds. Or
moving files you want to "save" so that a "make install" can succeed just

>     to have to move those saved files back again as root overwriting the
newly installed files with the ones you wanted to keep etc..

>     You can always run the build script with a list of phases as options
to step through the install at your own pace. But then the ./build

>     script itself is open to change as well if you like.

    This is a great improvement to this system.  Matthias did a great job
engineering it.  However, as Linux and LFS grew and became more
sophisticated, the original hint became a little (?) pedantic and "brute
force." After a couple of builds, the interruptions were more than merely
frustrating.  That's why after I built my first {,B}LFS system, I generated
my own install.dir.lst.  I really want to see those 'root_*_install
commands."

>     Thanks again for your feedback, I hope I didn't miss addressing any
of your points.

    Thanks for your efforts.  You got them all. But.......I have some new
ones. :)



    I've beaten the "root_*" stuff enough.  In addition, however, I didn't
know that you could "su" in a script.  That makes things really great!



    I'm gonna go back to your build notes because I don't want to ask a
dumb question about build directories.  However, since you use 'ln -s' are
the xxxbuild and yyysrc directories just "dummies" or do they actually
exist? If they exist, what is their purpose?



    Last thing so you can continue to vacation.  I was interested to see
how you handled all the tar.{xz,gz,bz.bz2} files.  PATTERNS seems to be the
key.  In build.conf "PATTERNS='4.3.2.tar". I don't understand this.  Would
you explain it?  I hope this isn't going to be a "duh moment" for me. :)



    Thanks for getting back to me.  Enjoy the sun and fun.  In another
venue I'll tell you about the "Kalakua Death March" I learned in the Navy.



    You take care too,

    Dan



    --

    http://linuxfromscratch.org/mailman/listinfo/lfs-support

    FAQ: http://www.linuxfromscratch.org/lfs/faq.html

    Unsubscribe: See the above information page
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfromscratch.org/pipermail/lfs-support/attachments/20131007/4ee68237/attachment.html>


More information about the lfs-support mailing list