Part 2: Packaging with Packaging Utility
- Published:
- categories: ibm
- tags: software, packaging-utility
IBM Packaging Utility is a helper to the IBM Installation Manager tool as it is used to manipulate Installation Manager Repositories. When creating installation media, the common use case is to include a bootstrapping version of Installation Manager and bundles Installation Manager Repositories to install from.
Goals
Let’s start out with why would you bother doing this, first it will save time, second it improves the predictability of installations, and third it means that you can facilitate automating these installs in a meaningful way. No it is not required, but I have run literally hundreds of installations manually silently, over the phone, via remote desktop, and each one has required a certain amount of my brain power and more importantly time to set the context, figure out where everythint is and worst of all troubleshooting why it is not working or what is missing. By using the standard IBM tooling for doing this you can at least gain some assurances that the tools can do the base tasks of making sure you have valid install media, centralize what end users are installing from, and stop wasting so much space hosting these things in 15 different locations.
Reading through and executing this should take you no more than an hour to do, and if you keep all the media and repos local for your testing you will be able to test and verify this process quickly and easily.
Find your pucl command
On my mac this is here, I will just create a helper variable so I can use the $pucl from now on. (Had issues with my alias not being used when I sudo to root, so while it is a bit clunkier the variable just seems to work).
export pucl=/Applications/IBM/PackagingUtility/PUCL
Now confirm you can run the command and we are ready to move on.
sgwilbur@gura:~$ sudo ${pucl} -h
Password:
help, -help, -h, -?, ? (all, <command>)
Displays short help about available commands for PUCL or optionally more detailed help about a specific command.
copy <id>[_<version>]
Copy a package or fix to a repository.
delete <id>_<version>
Delete a package or fix from a repository.
listAvailableFixes <id>_<version>
List information about available fixes.
listAvailablePackages
List information about available packages.
-masterPasswordFile <masterPasswordFilePath>
Defines master password file
-nl <nl>
Specify desired language to be used.
-secureStorageFile <secureStorageFilePath>
Defines secure storage file
-showProgress, -sP
Show progress.
-showVerboseProgress, -sVP
Show verbose progress.
Using the Packaging Utility
Now lets use the packaging utility, I’ve downloaded all the server components for Rational Test Workbench and I want to package them up into a single repository to simplify the management and make silent installs easier, as in part 1.
Prepare my environment and get some media ready to go:
This is a step you would normally do for each install, so nothing new here except may be for how I choose to lay it out.
sgwilbur@gura:~$ mkdir /tmp/PU-testing && cd /tmp/PU-testing
sgwilbur@gura:/tmp/PU-testing$ mkdir archives extracts repo
... <download my media to archives ...
sgwilbur@gura:/tmp/PU-testing$ for i in archives/*.zip; do unzip -o $i -d extracts/; done
... lots of scrolling file names ...
sgwilbur@gura:/tmp/PU-testing$ du -sh *
6.4G archives
7.1G extracts
0B repo
sgwilbur@gura:/tmp/PU-testing$ tree -L 2
.
├── archives
│ ├── IBM_RIT_AGENT_V8.5_MP_ML.zip
│ ├── IBM_RIT_V8.5_CONTROL_PANEL_MP_ML.zip
│ ├── IBM_RIT_V8.5_PLATFORM_PACK_MP_ML.zip
│ ├── IBM_RTVS_V8.5_SETUP_MP_ML.zip
│ └── RPT_Agent_Controller_8.5_Part1.zip
├── extracts
│ ├── RITA_SETUP
│ ├── RITPP_SETUP
│ ├── RPTAGENT_SETUP
│ ├── RTCP_SETUP
│ └── RTVS_SETUP
└── repo
8 directories, 5 files
sgwilbur@gura:/tmp/PU-testing$ tree -L 3 -d
.
├── archives
├── extracts
│ ├── RITA_SETUP
│ │ └── disk1
│ ├── RITPP_SETUP
│ │ └── disk1
│ ├── RPTAGENT_SETUP
│ │ └── disk1
│ ├── RTCP_SETUP
│ │ └── disk1
│ └── RTVS_SETUP
│ ├── InstallerImage_aix_gtk_ppc
│ ├── InstallerImage_linux_gtk_x86
│ ├── InstallerImage_linux_gtk_x86_64
│ ├── InstallerImage_solaris_gtk_sparc
│ ├── InstallerImage_win32_win32_x86
│ ├── InstallerImage_win32_win32_x86_64
│ ├── documentation
│ ├── launchpad
│ └── msd
└── repo
21 directories
sgwilbur@gura:/tmp/PU-testing$
We can imagine that this process would normally be repeated a few dozen times if each person does it, or a better scenario the first user does it and hosts it on a share for others to use. Depending on your team size and org support you may take this further, but I want to show you from here how you could spend an extra 30-60m the first time to save a number of hours later. Not only hours spent doing the same steps manually, but downloading, copying, and correcting errors when unavoidably someone installs an incorrect version from the old share…
At this point this process should have taken between 15-60m depending on your network and context switching you do while waiting.
Inspecting repositories:
Now we have some open repositories to inspect and work with, let’s figure out what how we go about finding what is in them. First I setup a variable so I don’t need to keep typing the path over and over, since I am lazy you won’t see it depending your shell my OS X bash shell turns it to the value when I type $e/<TAB>, then we start playing around.
sgwilbur@gura:/tmp/PU-testing$ export e=/tmp/PU-testing/extracts/
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl help listAvailablePackages
listAvailablePackages
List information about available packages.
-connectPassportAdvantage
Append the PassportAdvantage repository to the repository list.
-features
Include information about features in the output.
-long
Include more details in the output.
-masterPasswordFile <masterPasswordFilePath>
Defines master password file
-preferences <key>=<value>(,<key2>=<value2>...)
Specify a preference value or a comma-delimited list of preference values to be used.
-prompt
Display prompts for passwords or to insert disks on the console.
-repositories <repository>(,<repository2>...)
Specify repositories to be used.
-secureStorageFile <secureStorageFilePath>
Defines secure storage file
-showPlatforms
Include supported platforms in the output.
-showUpdateFromVersion
*** Technology Preview ***
Include the package version that can be updated in the output.
-useServiceRepository
Specify to search service repositories.
Print information to the console about the available packages.
If the -features option is used, information about the features is printed.
If the -long option is used, the following information is printed:
<repository> : <package id>_<package internal version> : <package name> : <package version>
Examples:
listAvailablePackages -long -repositories https://example.com/IBMIMrepository
Looks pretty straight forwards let’s try it out:
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl listAvailablePackages -long -repositories extracts/RITA_SETUP/disk1
CRIMC1011W WARNING: The following repositories are not connected:
WARNING: Failed to connect repository 'extracts/RITA_SETUP/disk1' either because the repository does not exist, the repository is closed or the repository server can not be reached.
WARNING: Some reasons why a repository connection fails are:
WARNING: Is the Repository location mistyped?
WARNING: Is the local network down?
WARNING: If you use a firewall, does it allow the Installation Manager to access the server?
WARNING: If your computer uses a proxy server, are the proxy settings for the Installation Manager set correctly?
Ok first snag, the paths need to be absolute, no reference to this in the documentation or help, but whatever let’s keep going.
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl listAvailablePackages -long -repositories /tmp/PU-testing/extracts/RITA_SETUP/disk1/
/tmp/PU-testing/extracts/RITA_SETUP/disk1/ : com.ibm.rational.rita.offering_8.5.0.I20130529_1634 : IBM? Rational? Integration Tester Agent : 8.5
Immediately sick of typing out all those repositories, I type them all out once more and export it as a variable:
sgwilbur@gura:/tmp/PU-testing$ export repos=/tmp/PU-testing/extracts/RITA_SETUP/disk1/,/tmp/PU-testing/extracts/RITPP_SETUP/,/tmp/PU-testing/extracts/RPTAGENT_SETUP/,/tmp/PU-testing/extracts/RTCP_SETUP/disk1/,/tmp/PU-testing/extracts/RTVS_SETUP/
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl listAvailablePackages -long -repositories $repos
/tmp/PU-testing/extracts/RTVS_SETUP/ : com.ibm.rational.performance.tester.agent_8.5.0.RPTAO85-I20130531_1322 : IBM? Rational? Performance Tester Agent : 8.5
/tmp/PU-testing/extracts/RTVS_SETUP/ : com.ibm.rational.rita.offering_8.5.0.I20130529_1634 : IBM? Rational? Integration Tester Agent : 8.5
/tmp/PU-testing/extracts/RTVS_SETUP/ : com.ibm.rational.ritpp.offering_8.5.0.I20130529_1634 : IBM? Rational? Integration Tester Platform Pack : 8.5
/tmp/PU-testing/extracts/RTVS_SETUP/ : com.ibm.rational.rtcp.offering_8.5.0.I20130529_1634 : IBM? Rational? Test Control Panel : 8.5
/tmp/PU-testing/extracts/RITA_SETUP/disk1/ : com.ibm.rational.rita.offering_8.5.0.I20130529_1634 : IBM? Rational? Integration Tester Agent : 8.5
/tmp/PU-testing/extracts/RITPP_SETUP/ : com.ibm.rational.ritpp.offering_8.5.0.I20130529_1634 : IBM? Rational? Integration Tester Platform Pack : 8.5
/tmp/PU-testing/extracts/RPTAGENT_SETUP/ : com.ibm.rational.performance.tester.agent_8.5.0.RPTAO85-I20130531_1322 : IBM? Rational? Performance Tester Agent : 8.5
/tmp/PU-testing/extracts/RTCP_SETUP/disk1/ : com.ibm.rational.rtcp.offering_8.5.0.I20130529_1634 : IBM? Rational? Test Control Panel : 8.5
Ok and I am seeing that the RTVS_Setup image is a special type that reads the images around it based on their extract layout, in this case it just repeats the others, so let’s you can leave it, drop it, or only include that one and we get back to a simpler view:
sgwilbur@gura:/tmp/PU-testing$ export repos=/tmp/PU-testing/extracts/RITA_SETUP/disk1/,/tmp/PU-testing/extracts/RITPP_SETUP/,/tmp/PU-testing/extracts/RPTAGENT_SETUP/,/tmp/PU-testing/extracts/RTCP_SETUP/disk1/
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl listAvailablePackages -long -repositories $repos
/tmp/PU-testing/extracts/RITA_SETUP/disk1/ : com.ibm.rational.rita.offering_8.5.0.I20130529_1634 : IBM? Rational? Integration Tester Agent : 8.5
/tmp/PU-testing/extracts/RITPP_SETUP/ : com.ibm.rational.ritpp.offering_8.5.0.I20130529_1634 : IBM? Rational? Integration Tester Platform Pack : 8.5
/tmp/PU-testing/extracts/RPTAGENT_SETUP/ : com.ibm.rational.performance.tester.agent_8.5.0.RPTAO85-I20130531_1322 : IBM? Rational? Performance Tester Agent : 8.5
/tmp/PU-testing/extracts/RTCP_SETUP/disk1/ : com.ibm.rational.rtcp.offering_8.5.0.I20130529_1634 : IBM? Rational? Test Control Panel : 8.5
Ok now we can inspect local repos, what about remote ones, sure enough same command works with urls, cool!
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl listAvailablePackages -long -repositories http://mediaserver/repos/urbancode/release/6.0.1.0/
http://mediaserver/repos/urbancode/release/6.0.1.0/ : com.ibm.urbancode.urelease_6.0.1.UCRELO6001-I20131011_1626 : IBM UrbanCode Release : 6.0.0.1
Or even better a remote repo that requires authentication:
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl listAvailablePackages -long -repositories https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/
CRIMC1011W WARNING: The following repositories are not connected:
WARNING: Failed to connect to repository 'https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/'.
WARNING: Missing credentials for prompt: Credentials are required to connect to jazz.net: "Jazz.net Community Site"
WARNING: Use the -prompt option to receive a credential prompt. The credentials are saved to the secure storage file.
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl listAvailablePackages -long -repositories https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ -prompt
=====> PUCL> Password required
Credentials are required to connect to jazz.net: "Jazz.net Community Site"
Select:
P. Provide credentials and connect
C. Cancel
Select 'P' to enter credentials and connect, or 'C' to cancel.
-----> P
Enter user name:
-----> swilbur
Enter password:
To skip entering password, press Enter. In most cases, you must enter a password to access a repository.
----->
1. [ ] Save password if valid
Saved passwords are stored on your computer in a file which is difficult, but not impossible, for an intruder to read.
O. OK, C. Cancel
-----> 1
1. [X] Save password if valid
Saved passwords are stored on your computer in a file which is difficult, but not impossible, for an intruder to read.
O. OK, C. Cancel
-----> O
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.com.ibm.team.concert.dotnet_4.0.6.20140127_0614 : Rational Team Concert - Client for Microsoft Visual Studio IDE : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.com.ibm.team.concert.msscci_4.0.6.20140127_0956 : Rational Team Concert - MSSCCI Provider : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.com.ibm.team.concert.winshell_4.0.6.20140127_0814 : IBM Rational Team Concert Shell : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.team.install.jfs.app.jts_4.0.6000.RJF-I20140130-1407-r406- : Jazz Team Server : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.team.install.jfs.app.product-clm_4.0.6000.CALM-I20140205-1923-r406 : Trial keys for Collaborative Lifecycle Management Products : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.team.install.jfs.app.rdm_4.0.6000.RMS4_0_6-I20140203_1432 : Requirements Management : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.team.install.jfs.app.rqm_4.0.6000.RQMMainline-I20140205_1831 : Quality Management : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.team.install.jfs.app.rtc_4.0.6000.RTC-I20140205-1437-r406 : Change and Configuration Management : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.team.install.rtc.buildsystem_4.0.6000.RTC-I20140205-1437-r406 : Rational Team Concert - Build System Toolkit : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.team.install.rtc.ccconnector_4.0.6000.RTC-I20140205-1437-r406 : Rational Team Concert - ClearCase Synchronizer : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.team.install.rtc.client.eclipse_4.0.6000.RTC-I20140205-1437-r406 : Rational Team Concert - Client for Eclipse 3.6.x IDE : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.team.install.rtc.client.eclipse.ies4_4.0.6000.RTC-I20140205-1437-r406 : Rational Team Concert - Client for Eclipse 4.2.x IDE : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.team.install.rtc.client.extension_4.0.6000.RTC-I20140205-1437-r406 : Rational Team Concert - Client Extension for Eclipse 3.x : 4.0.6
https://jazz.net/downloads/clm/4.0.6/4.0.6/install-repository/clm-offerings/repository/ : com.ibm.team.install.rtc.cqconnector_4.0.6000.RTC-I20140205-1437-r406 : Rational Team Concert - ClearQuest Synchronizer : 4.0.6
Ok now we have all the tools we need to start composing our own IM repositories, this has only taken another 5-15m of playing around to get a handle on what is going on here, next we will do something with it.
Creating our own IM Repositories
Let’s again use the $pucl help command to figure out what we are doing:
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl help copy
copy <id>[_<version>]
Copy a package or fix to a repository.
<id>[_<version>]
Specify a package or a version of a package. If the version is not specified, the latest version of that package will be copied. Fix id and version can also be used to copy a specific fix.
-acceptLicense
Indicate acceptance of the license agreement.
-connectPassportAdvantage
Append the PassportAdvantage repository to the repository list.
-masterPasswordFile <masterPasswordFilePath>
Defines master password file
-platform os=<os> | arch=<arch> | os=<os>,arch=<arch>
Specify the platform.
-preferences <key>=<value>(,<key2>=<value2>...)
Specify a preference value or a comma-delimited list of preference values to be used.
-preview
Instead of actually performing the operation, request a preview of the result.
-prompt
Display prompts for passwords or to insert disks on the console.
-repositories <repository>(,<repository2>...)
Specify repositories to be used.
-secureStorageFile <secureStorageFilePath>
Defines secure storage file
-target <target repository>
Specify the target repository.
-updateFromVersion <version>
*** Technology Preview ***
Specify the version that you are updating from.
-useServiceRepository
Specify to search service repositories.
Copy a package or fix to a repository. The target repository must be specified by using the -target option with one or more of the following options:
-connectPassportAdvantage
-repositories
-useServiceRepository
Examples:
copy packageid_version -repositories https://sourcerepository -target c:\mydir
Again seems to make sense, let’s give it a try:
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl copy "com.ibm.rational.rita.offering_8.5.0.I20130529_1634" -repositories $repos -target repo
CRIMG1000E ERROR: You must accept the license agreements for package com.ibm.rational.rita.offering.
Use -acceptLicense argument to accept the license.
The operation failed to complete.
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl copy "com.ibm.rational.rita.offering_8.5.0.I20130529_1634" -repositories $repos -target repo -acceptLicense
The operation completed successfully.
Cool, now let’s see what it did:
sgwilbur@gura:/tmp/PU-testing$ du -sh *
6.4G archives
7.1G extracts
0B repo
Ah wtf, where did it go ? so I search for it, and I’ll just get rid of it we can do it again:
sgwilbur@gura:~$ sudo find / -name repo
/Applications/IBM/PackagingUtility/repo
sgwilbur@gura:~$ sudo du -sh /Applications/IBM/PackagingUtility/repo
1.3G /Applications/IBM/PackagingUtility/repo
sgwilbur@gura:~$ sudo rm -rf/Applications/IBM/PackagingUtility/repo
Obviously it was created in the repo directory relative to where the Package Utility starting from (as sarcastically as I can say it), should have learned my lesson earlier, to just use the absolute path.
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl copy "com.ibm.rational.rita.offering_8.5.0.I20130529_1634" -repositories $repos -target /tmp/PU-testing/repo -acceptLicense
The operation completed successfully.
sgwilbur@gura:/tmp/PU-testing$ du -sh *
6.4G archives
7.1G extracts
1.3G repo
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl listAvailablePackages -long -repositories /tmp/PU-testing/repo
/tmp/PU-testing/repo : com.ibm.rational.rita.offering_8.5.0.I20130529_1634 : IBM? Rational? Integration Tester Agent : 8.5
Well isn’t that special! (in your best churchlady voice).
So again another 5-15m trying to digest how this works and actually running it.
Bringing it all together with a bit of scripting magic
Could just run this on the command line, but it is simpler and easier to just put it in a script that can be shared and more importantly versioned, check it out bundle-repos.sh
#!/bin/bash
# Helper script for bundling individual IM repos into custom IM repos
# @sgwilbur
pucl=/Applications/IBM/PackagingUtility/PUCL
work_dir=/tmp/PU-testing
extracts_dir=${work_dir}/extracts
repos=${extracts_dir}/RITA_SETUP/disk1/,${extracts_dir}/RITPP_SETUP/,${extracts_dir}/RPTAGENT_SETUP/,${extracts_dir}/RTCP_SETUP/disk1/
target_repo=/tmp/PU-testing/repo
for i in `$pucl listAvailablePackages -long -repositories $repos | awk {'print $3'}`
do
echo "Copying ${i} to ${target_repo}"
sudo $pucl copy "${i}" -repositories ${repos} -target ${target_repo} -acceptLicense
done
Please rememeber to do this locally, I have waited over too many shoulders while well meaning people have told me how fast their company network is just to wait 30m for a stupid UNC file copy… Here is an example of why you should do this locally, less than a minute then I am free to copy this directory where ever it needs to go just a slow as I like and not block a program, cpu, and network i/o.
sgwilbur@gura:/tmp/PU-testing$ sudo rm -rf repo/*
sgwilbur@gura:/tmp/PU-testing$ time sudo ./bundle-repos.sh
Copying com.ibm.rational.rita.offering_8.5.0.I20130529_1634 to /tmp/PU-testing/repo
The operation completed successfully.
Copying com.ibm.rational.ritpp.offering_8.5.0.I20130529_1634 to /tmp/PU-testing/repo
The operation completed successfully.
Copying com.ibm.rational.performance.tester.agent_8.5.0.RPTAO85-I20130531_1322 to /tmp/PU-testing/repo
The operation completed successfully.
Copying com.ibm.rational.rtcp.offering_8.5.0.I20130529_1634 to /tmp/PU-testing/repo
The operation completed successfully.
real 0m51.572s
user 0m55.551s
sys 0m8.917s
sgwilbur@gura:/tmp/PU-testing$ du -sh *
6.4G archives
4.0K bundle-repos.sh
7.1G extracts
2.7G repo
sgwilbur@gura:/tmp/PU-testing$ sudo $pucl listAvailablePackages -long -repositories /tmp/PU-testing/repo
Password:
/tmp/PU-testing/repo : com.ibm.rational.performance.tester.agent_8.5.0.RPTAO85-I20130531_1322 : IBM? Rational? Performance Tester Agent : 8.5
/tmp/PU-testing/repo : com.ibm.rational.rita.offering_8.5.0.I20130529_1634 : IBM? Rational? Integration Tester Agent : 8.5
/tmp/PU-testing/repo : com.ibm.rational.ritpp.offering_8.5.0.I20130529_1634 : IBM? Rational? Integration Tester Platform Pack : 8.5
/tmp/PU-testing/repo : com.ibm.rational.rtcp.offering_8.5.0.I20130529_1634 : IBM? Rational? Test Control Panel : 8.5
Conclusion
You now have a way to save space, simplify the install process(save time), and unify the storage and distribution mechanism for software that requires IBM Installation Manager to install, which includes most Rational, some WebSphere, and other IBM software for enterprise deployments. Ultimately this improves the tools and IT admins lives in this specific area, and every little bit helps.
Since I found the Enterprise Deployment tools I have been playing with them and using them to my advantage, I hope that some of this helps you do the same. If you have questions or problems executing these commands or you are on Windows and some of this does not apply feel free to comment and let me know what did and did not work for you.
I’ll post once more on this topic on how to share them once you are done.