Implementing new ASDD nodes: Isite on Windows NT

Documentation is under development

Overview

This documentation is incomplete.

Isite was originally developed by Center for Networked Information Discovery and Retrieval (CNIDR) and is now managed by A/WWW Enterprises

• index, search, present XML document collections
• can connect to a separate program to present a full metadata record
• various other structured text data formats ... DIF, colon-delimited, HTML, email archives

See the file RELEASE.MSDOS in the Isite distribution.

Documentation references

Further information is available at ...

• A/WWW Enterprises
• Center for Networked Information Discovery and Retrieval (CNIDR)
• Etymon Systems
• Year 2000 compliance
• FGDC tutorial: Set up a Clearinghouse node

Download the software

The current Isite2 is available at A/WWW Enterprises (follow the link from the home page to "Isite Distribution", then to the "Isite2" directory).

Unpack the software into a preparation directory on your system. Follow the Isite2 instructions if you are building from source.

ASDD customisation

Configuration files and the Isearch "doctypes" called "anzmeta" for indexing and searching ANZLIC metadata documents have already been added to the Isite2 distribution.

Create some directories

Set up the following directories. They can be anywhere on your server and do not need to be under the HTTP server documents root directory. Probably in a different place to your current installation ...

• $ISITE_BUILD ... where you unpacked the Isite distribution
• $ISITE_BIN ... where Isite binaries and configuration files will be installed
• $ISITE_CONF ... you might prefer to keep the configuration files separate to the binaries
• $ISITE_DATA ... where the collections of metadata documents are located
• $ISITE_DB ... where Isite databases will be built during indexing

Compiling

To compile and build Isite on Windows NT you will need to have the Microsoft "Visual Studio C++" compiler installed.

• change directory to where you unpacked Isite ($ISITE_BUILD)
• follow the instructions in the file RELEASE.MSDOS in the Isite distribution.
• the binaries that are produced will be in the $ISITE_BUILD/exe directory

Installation

Installation is currently a manual process.

Copy the following binaries from $ISITE_BUILD/exe to $ISITE_BIN ...

• zserver ... the Isite Z39.50 server
• zbatch ... command-line client for conducting batch queries
• zping ... command-line client for testing if a server is alive
• zclient ... command-line client for conducting queries
• izclient ... interactive text-based Z39.50 client
• Iindex ... used to build the Isite index databases
• Isearch ... command-line search client
• Iutil ... utility for interrogating Isite databases

Copy all of the configuration files from $ISITE_BUILD/conf/anzmeta to $ISITE_CONF. They will be explained in the next section.

Configuration

There are example configuration files included in the Isite2 distribution at ./conf/anzmeta/

The only two configuration files that should differ from the distribution are sapi.ini and zserver.ini. All configuration files are described below ...

sapi.ini

• Defines attributes of each document collection that is available to Isearch.
• Defines the location of the database on the file system.
• Identifies which method to use to search the collection:
  • Keyword ISEARCH says to use this Isearch database that was created by running Iindex (which used the relevant Isearch doctype to parse the structured data files).
• Defines which mapping files (see below) to use with the collection.
• Example sapi.ini

zserver.ini

• Used for initialisation when the Zserver is started.
• Defines which databases (that are described in sapi.ini) are to be mounted and made available for searching via the Zserver.
• You could define a different port number in this file if you want to leave your existing Zserver running.
• Example zserver.ini

anzlic.fields

• The Isearch doctype uses this file to define the data type of each particular metadata element (field) within the XML document.
• The anzlic.fields file is specified in the Iindex command at indexing time.
• Values: num (numeric fields e.g. <northbc>), date (single date e.g. <begdate>, <metd>), date-range (a range of dates i.e. <timperd>), gpoly (greater bounding polygon e.g. <bounding>).
• Any field that is not defined in this configuration file is assumed to be a text field.
• Example anzlic.fields

"Use Attribute" maps

• Z39.50 insulates the user from the actual schema (structure and field names) of each document management system.
• These configuration files enable the standard interface "Use Attribute" numbers (the numbers behind the WWW interface pick-list names) to be mapped to the actual field names in each particular document collection.
• Each collection of documents can have various different mapping files.
• These mapping files are specified in the sapi.ini configuration file.
• Examples for use with the Isearch doctype "anzmeta" for ANZMETA v1.3 XML files
  • anzmeta.geomap for GEO Attribute Set
  • anzmeta.bibmap for BIB-1 Attribute Set
  • anzmeta.gilsmap for GILS Attribute Set

Prepare the metadata collection

Every node must be able to present XML, HTML and plain text (SUTRS) versions of the metadata. Geospatial metadata management facilities are used to prepare, store, and present the metadata.

The Z39.50 server can conduct searching and presentation using either a collection of XML metadata documents or connecting to a database or other repository.

The structured metadata documents or XML files must conform to the ANZMETA Document Type Definition (DTD) v1.1, v1.2 or v1.3 and should have .xml filename extension to allow the Z39.50 server to present the XML file if requested by a capable client.

The document collection will be indexed by an Isearch "doctype".

Index the metadata

This section explains how to index a collection of XML metadata documents. If you are using a database to store and search your metadata, then you need to configure relational database access.

The "Isearch" component of Isite uses software called "doctypes" to read and interpret the XML metadata documents. The doctypes have dual roles: to index the metadata to create a searchable database, and to conduct searching and present the results in whatever form that is requested by the client.

Each collection of dataset descriptions has three files for each dataset description ...

• basename.xml ... the structured metadata (could have .xml extension)
• basename.html ... the HTML file which is used at presentation time (could have .htm or .html extension)
• basename.txt ... the plain text file or SUTRS which is used at presentation time

Below are example Iindex commands to prepare a searchable database of your metadata. You could place the command in a shell script. The examples assume that all documents are in one data directory. If your data is in separate directories then you could use the UNIX commands "find" and "sed" to automatically prepare the list of pathnames to feed into Iindex.

To index a collection of XML files that conform to the ANZMETA Document Type Definition (DTD) v1.1, v1.2 or v1.3 ...

# the name for your index database of dataset descriptions
DB_NAME=test1

# run Iindex to parse the XML files
# using the Isearch doctype called "anzmeta" 
$ISITE_BIN/Iindex -d $ISITE_DB/$DB_NAME -t anzmeta -m 4 \
-o fieldtype=$ISITE_BIN/anzlic.fields $ISITE_DATA/*.xml

Start the Z39.50 server

Start the Zserver by issuing the following command. The server will start up, mount the specified databases, and then listen on the specified port. You could place this command in a shell script. You may also want to redirect the output to a logfile if you want detailed connection and searching information. The standard logfile (specified in zserver.ini) collects only very basic connection information.

# start the Z39.50 server and run it in the background
$ISITE_BIN/zserver -i$ISITE_BIN/zserver.ini &

Testing your node

The best way to test your node is to use the Isite program called "zbatch" which allows you to specify a set of queries in a plain text file. Zbatch will connect to the specified server and run the queries sequentially.

See the document Testing Isite ASDD nodes.

Hosted collections

Any one node can also host a collection of geospatial metadata for another organisation. In that way, an organisation that does not have an actual Z39.50 server can appear to be a fully-fledged node of the ASDD. The collection of XML documents and the corresponding presentation documents need to be on the same machine on which the Zserver is running. Simply index the collection of documents using Iindex and define all of the collections in the sapi.ini and zserver.ini configuration files.

Register your node

Follow the instructions to register your new node with the ASDD gateway WWW interface.