Installing the 2MASS database

Instructions for installing a copy of the 2MASS database for the EuroVo^? DataCentre^? workshop.

-- DaveMorris - 24 Jun 2005

Installing PostgreSQL

The PostgreSQL server is available as a rpm install from the default yum repository.

To install the server, login as root and run the following.

    yum install postgresql-server

This should download the required packages and prompt you before the install.

    [root@host path]#
    [root@host path]# yum install postgresql-server
    Setting up Install Process
    Setting up Repos
    livna-stable              100% |=========================|  951 B                    00:00
    fedora-updates            100% |=========================|  951 B    00:00
    fedora-core               100% |=========================| 1.1 kB    00:00
    ....
    Dependencies Resolved
    Transaction Listing:
      Install: postgresql-server.i386 0:7.4.8-1.FC3.1 - fedora-updates
    Performing the following to resolve dependencies:
      Install: postgresql.i386 0:7.4.8-1.FC3.1 - fedora-updates
    Total download size: 5.0 M
    Is this ok [y/N]:

Say 'yes' and yum will perform the install :

    Is this ok [y/N]: y
    Downloading Packages:
    (1/2): postgresql-server- 100% |=========================| 3.0 MB    01:25
    (2/2): postgresql-7.4.8-1 100% |=========================| 2.0 MB    00:56
    Running Transaction Test
    Finished Transaction Test
    Transaction Test Succeeded
    Running Transaction
    Installing: postgresql 100 % done 1/2
    ....
    Installing: postgresql-server 100 % done 2/2
    ....
    Installed: postgresql-server.i386 0:7.4.8-1.FC3.1
    Dependency Installed: postgresql.i386 0:7.4.8-1.FC3.1
    Complete!

Starting PostgreSQL

Once the install has completed, start the postgreSQL server using the init script.

    /etc/init.d/postgresql start

The first time this is run it should initialise the database and then start the server.

    [root@host path]#
    [root@host path]# /etc/init.d/postgresql start
    Initializing database:                                     [  OK  ]
    Starting postgresql service:                               [  OK  ]

Creating the database

The data install scripts need to be run from the postgres user account. Before you login as postgres, create a directory to download the data files to, and change the permissions to allow read/write to all.

    [root@host path]#
    [root@host path]# mkdir twomass

    [root@host path]#
    [root@host path]# chmod a+rw twomass

    [root@host path]#
    [root@host path]# cd twomass/

In order to create new tables in the database, you will need to su to the postgres user account.

    [root@host twomass]#
    [root@host twomass]# su postgres
    bash-3.00$

The install-twomass.sh shell script perfoms following tasks.

• init Download and install the database table schema.
• first Download and install the first of the data files.
• test Runs a simple test query to count the rows.
• rest Download and install the rest of the data files.
• index Create the main table indexes.

To install the database table schema, run the init command.

    ./install-twomass.sh init

This should download the database schema and install it.

    bash-3.00$
    bash-3.00$ ./install-twomass.sh init

    Fri Jun 24 16:44:49 BST 2005
    Downloading schema ...

    Fri Jun 24 16:44:49 BST 2005
    Downloading ...
        File : twomass_psc_schema
        Path : ftp://ftp.ipac.caltech.edu/pub/2mass/allsky/twomass_psc_schema
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100  1288  100  1288    0     0    475      0  0:00:02  0:00:02 --:--:--     0

    Fri Jun 24 16:44:52 BST 2005
    Creating database ...

    You are now connected to database "template1" as user "postgres".
    CREATE DATABASE
    You are now connected to database "wsdb" as user "postgres".
    CREATE TABLE

    Fri Jun 24 16:44:53 BST 2005
    Database table created ...

Installing the data

To load the first data file run the first command.

    ./install-twomass.sh first

This will download and install the first of the data files.

    bash-3.00$ ./install-twomass.sh first

    Fri Jun 24 16:45:09 BST 2005
    Downloading ...
        File : psc_aaa.gz
        Path : ftp://ftp.ipac.caltech.edu/pub/2mass/allsky/psc_aaa.gz
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100  470M  100  470M    0     0   393k      0  0:20:24  0:20:24 --:--:--  411k

    Fri Jun 24 17:05:34 BST 2005
    Installing ...
        File : psc_aaa.gz

    Fri Jun 24 17:15:12 BST 2005
    Data file installed

To check that the data was loaded correctly, run the test command

    ./install-twomass.sh test

This runs a simple query to count the number of rows in the database table.

    bash-3.00$
    bash-3.00$ ./install-twomass.sh test

    Fri Jun 24 17:17:16 BST 2005
    Running test query ...
      count
    ---------
     5144500
    (1 row)

Assuming the first download worked, we can now download and install the rest of the data files.

WARNING This will take quite a while, there are 94 data files to load.

    ./install-twomass.sh all

This will download and install the rest of the data files from the caltech server.

    bash-3.00$
    bash-3.00$ ./install-twomass.sh all

    Fri Jun 24 17:21:17 BST 2005
    Downloading ...
        File : psc_aab.gz
        Path : ftp://ftp.ipac.caltech.edu/pub/2mass/allsky/psc_aab.gz
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
        ....

Creating the indexes

Once all of the data files have been loaded, use the index command to create the main indexes.

    ./install-twomass.sh index

This will create the two main indexes on the ra and decl columns.

    bash-3.00$
    bash-3.00$ ./install-twomass.sh index

    Creating indexes ...
    CREATE INDEX
    CREATE INDEX
    Indexes created ...