|
Modified: 2004-08-17
Establishing individual ASDD nodes
This document is sponsored by the Queensland Spatial
Information Infrastructure Strategy (QSIIS)
Shows how any organisation can participate in the ASDD.
Explains the operation of the ASDD and
where individual document collections fit within it. Provides an
overview of how to go about establishing
your presence in the ASDD and thence other networked information
systems. Some case studies will give a
general idea of how other organisations have done it and what
resources are involved.
This document is intended to provide high-level explanation
only, giving a guide as to costs and effort and how to do it. You
will find direct links to relevant ASDD technical documentation
for further detail.
You have decided to establish your own node of the ASDD,
rather than simply add your documents into your state-level
general node. Essentially, you are going to set up a Z39.50
search server to provide search and presentation services for
your metadata collection. There is no user-interface component,
as that is provided by gateways such as ASDD
and FGDC.
Your custodian metadata collections are the most
important component, together with the reliable
discovery and presentation of your documents. This will be
achieved quite simply by managing your metadata according to the
relevant standards and by providing search facilities using
correctly configured capable software. The case
studies below will provide some examples of how other
custodians have built their part of the ASDD.
- you already gather and manage the metadata
- as a separately funded ongoing task
- this is such an important issue that it is also referred to
here
- you have already liaised with your state coordination people
- they need to know that you intend to set up a node
- they can assist you with various advice
- contact details are provided in the "How to" list
below
- you intend to establish your own node of the ASDD
- as opposed to simply adding your documents to your
state-level jurisdictional node
- would expect to have a reasonable sized collection of
documents
The term "organisation" is used to mean any
government agency, non-government organisation (NGO), commercial
business, industry organisation, scientific research facility,
educational institution, community group ... any group that meets
the requirements for participation can
establish an ASDD presence.
Here are some dot points that you should consider. Of course,
some items are en twined with others, not every task is listed,
and these are not necessarily in sequence. If you do not
understand the ASDD operating environment and how you fit into
it, then read the overview below. Jargon
alert ! ... the terminology section at
the bottom will help.
- Assign staff to the task
- metadata champion and coordinator
- technical person as Z39.50 manager (if you decide to run your
own Zserver)
- project manager
- Develop a strategy and overall plan
- Liaise with ASDD
coordination
- Review available ASDD technical
documentation
- provides overview and background
- has specific discussion on key topics
- provides outlines of certain procedures
- Decide on a metadata management solution
- some options for geospatial
metadata management are described
- may need a combination of solutions
- this is one of your big decisions
- the case studies provide some help
- the key requirement is the delivery of metadata documents
from your server that are well-structured, reliable, and comply
with the standards
- Validate your metadata
- ensure professional and reliable ASDD presence
- XML metadata documents must conform to the relevant standards (ANZMETA DTD and ANZLIC Metadata
Guidelines)
- automated metadata validation facilities are available or
build your own
- Decide whether to run your own Z39.50 server or use a hosted
node
- installing and managing your own server will of course
require expertise and resources
- or you could simply have a hosted node behind some other
organisation's Z39.50 server (it looks like it is yours)
- dataset descriptions must be unique and live at only one ASDD
node
- Decide how to store your metadata and provide search access
- XML document collection using the computer's file system,
i.e. individual documents as files
- or database capable of supporting multiple simultaneous
Internet access
- warning: if you use a database to manage
your metadata then you would be better to regularly dump actual
documents from the database - otherwise you would need to build
programs that use SQL to interrogate your database on behalf of
the Z39.50 server, which is difficult, and anyway database
full-text search is limited
- whether to use static presentation documents or on-the-fly
generation
- we suggest that your document collection is stored beneath
the Web server document root, that way it will be available via
both the Z39.50 server and the Web server
- Decide whether to replicate your metadata collection on a
server outside your firewall
- most Z39.50 servers need to have the metadata documents
residing on the same computer as the Z39.50 server
- if you have a firewall, then you probably already have a Web
server outside the firewall - Z39.50 is no different, also a
read-only document delivery system
- Z39.50 servers must have no firewall restrictions, otherwise
no clients or gateways can connect to your server
- ensure that you establish a procedure for updating and
re-indexing the metadata (especially if it will reside on a
computer which is outside of your organisation)
- Decide which Z39.50 server software
- commercial and public-domains solutions are available
- some are listed at Implementing new ASDD
nodes
- if you have an existing Internet server then this choice will
be dependent on the operating system
- some Z39.50 services companies offer total solutions for
installation, configuration, and support
- Decide on computing platform for Z39.50 server
- the machine does not need to be a top-of-the-line server
- the Zserver will require similar resources to a Web
server
- Z39.50 solutions are available for both UNIX and Windows
NT
- Decide on Internet server location
- must be outside any firewall
- perhaps it will live at your Internet Service Provider (ISP)
premises
- in your overall costing do not forget to allow for server
hosting fees and data traffic charges
- also ensure that you will have login rights, to be able to
install and configure the software, and to be able to upgrade and
re-index the metadata from time-to-time
- Determine which port number
- 210 is official Z39.50 port
- or use any unpriveledged port (choose one in the range
5500-5555)
- ensure access through your firewall to this port
- Z39.50 traffic should be free from any firewall
restrictions
- Implement your Z39.50 server
- ASDD Technical Documentation: Implementing new ASDD
nodes provides some detail for a few of the possible server
solutions
- we suggest that you install and configure a server on your
internal network first, then implement your public server
- Test your Zserver
- run a test Z39.50 server on a different port to your
production node
- example batch testing procedure for Isite is documented
- use tools that come with your Z39.50 software, or build your
own from freely available toolkits
- there is a test gateway available at Geoscience Australia,
send feedback
to have your test node added
- test your node every time that you upgrade either the Z39.50
software or the metadata documents
- also conduct an outside test to ensure that the Zserver is
available through your firewall
- Describe your ASDD node
- add a high-level description of your document collection to
the list of ASDD nodes
- the node description gives users an overview of your document
collection, provides links back to specific areas of your web
site, advertises your Z39.50 connection details so that other
organisations can add your node to their gateways
- use the online "ASDD
node managers" application to consistently describe your
node and update the document whenever you choose
- technical Z39.50 node manager should join this document
management service
- also adds you to the technical discussion group and provides
access to some ASDD log files
- prepare documents in a private space and show to others for
approval
- press the "publish" button when your Z39.50 server
is tested and ready for searchers to access it
- this will send an email alert to the ASDD gateway manager at
Geoscience Australia to add your node to the search
interfaces
- Announce the availability of your new node
- tell people that you are now in the ASDD, encourage them to
discover your products
- Monitor the availability of your node
- ensure that it is always available
- utilise the ability of your server operating system to
regularly run a scheduled job to interrogate your server
- the automatic ASDD status
of nodes monitor will help you
- also review your server log files occasionally
Here are some brief case studies which show how other
organisations have done it. These documents speak in mythical
terms to give a general idea of what effort and resources are
involved. Remember, document management and information discovery
solutions have many permutations. These case studies try to give
some guidance.
- Case 1
- small organisation with a small metadata collection
- use existing Internet server machine
- use freely available Z39.50 software
- develop in-house expertise, do-it-ourself
- Case 2
- large organisation with several medium metadata
collections
- use existing Internet server machine
- all of our Internet services are managed by our Internet
Service Provider
- dump documents from database, validate and transfer out to
ISP
- use freely available Z39.50 software
- consultant does once-off installation and configuration, then
ISP maintains services
- Case 3 (not yet described)
- medium organisation with a medium metadata collection
- need new Internet server presence
- use commercial Z39.50 software with technical support
- Case 4
- small organisation with a small metadata collection
- hosted node behind another organisation's server
- minimal solution for now, we will grow later
Requirements are defined at
Requirements and Standards.
Here is some more detail ...
- Custodian collection of metadata
- XML document collection or a database capable of exporting
XML documents
- comply with ANZLIC criteria for ASDD
nodes
- any agency can participate
- Liaise with your state and national
coordination people
- Run a Z39.50 server
- commercial and public-domain solutions are available
- could instead have a hosted node behind an existing Z39.50
server
- respond to ASDD requirements
- present various views of metadata (HTML, text, XML)
- announce server to the various gateways
This is a brief overview of how the ASDD operates. See also
the ASDD architecture for
other explanations and ASDD
architecture diagram.
Many custodians each manage their own document collections and
update their metadata whenever they choose. Any custodian can
either establish their own search server (node) or can add their
documents to another relevant node.
Gateways provide search interfaces between the World Wide Web
and these Z39.50 search servers. There can be many different
gateways to same set of nodes or to a sub-set of the nodes.
Thematic gateways can present a topical look-and-feel to a
selected set of nodes.
The same Z39.50 node can serve many distinct directories, e.g.
the node at Australian Antarctic Data Centre (AADC) can
serve both the ASDD and Global Change Master Directory (GCMD) from the same
metadata repository. So AADC need only manage one metadata search
system.
Stand alone, reliable nodes are the key. Well-described and
well-structured metadata in your reliable ASDD node provides the
basis for cooperating in the various Australian spatial data
infrastructures (e.g. ASDI, QSIIS, ...). Other
applications can be build upon the well-structured metadata (e.g.
Web Mapping, online commerce, metadata that links directly to
product order forms).
The ASDD provides a well-structured foundation with consistent
catalogues of concise dataset descriptions. Future clearinghouses
for geospatial data will be built on top of this metadata (Z39.50
will inter-operate with future Open GIS Catalog Services).
The ASDD web site has plenty of technical
documentation to explain the detail. Here are some of the
most important documents ...
A short glossary of some of the most important terms. These
definitions are adapted from the Eco Companion Glossary.
- Client
- software program designed to interact with specific server
programs using accepted protocols
- your Netscape WWW browser is an example
- Dataset
- a particular coherent collection of data with a common
theme
- e.g. a listing of feral pest sightings
- geospatial datasets have reference to a location in
earth-centred space
- Document Type Definition (DTD)
- rules for structured documents
- To ensure that all dataset descriptions are of a
consistent type, the Document Type Definition (DTD) defines the
metadata elements, and their order, structure, and
relationships.
- A DTD allows different instances of documents of the same
type to be automatically processed in a uniform way.
- ANZMETA DTD defines geographic
dataset descriptions for Australia and New Zealand
- Gateway
- a door between the World Wide Web and the Z39.50 realm
- translate between the HyperText Transport Protocol of the WWW
and the Z39.50 search and retrieve protocol - HTTP to Z39.50
gateways
- allows you to use your Netscape WWW browser client to talk to
Z39.50 servers
- Geospatial metadata (dataset descriptions)
- concise descriptions of geospatial datasets (see
"Dataset " above)
- metadata is data about data
- Node
- a Z39.50 information server
- a searchable collection of geospatial metadata
- an information source in the ASDD
- Protocol
- a particular set of accepted rules of communication that
computers use to carry out a consistent dialogue over a
telecommunication connection
- HTTP protocol enables the World Wide Web
- Z39.50 protocol allows clients, servers, and gateways to
inter operate to discover and present structured information
- Repository
- the custodian's collection of metadata documents
- your Z39.50 search services will build a searchable ASDD node
by constructing a dictionary index from the document
repository
- Server
- two meanings:
- a machine that performs tasks and provides services to other
computers
- a program that provides services to client software
- a Web server runs on an Internet server machine and delivers
WWW documents requested by Netscape browser clients
- a Z39.50 server provides search and present services of any
number of document collections (called nodes)
- XML
- eXtensible Markup Language
- well-structured documents that conform to a set of rules
- separates the structure and content from the formatting and
presentation
- plain text documents that are independent of software,
vendors, proprietary formats, and computing platforms (XML
documents live longer)
- ANZMETA XML Document Type Definition
(DTD) defines the rules for geospatial metadata as XML
documents
- The XML Cover
Pages covers everything to do with SGML and XML
- Z39.50 search and retrieve protocol
- rules and generally accepted methods for information
retrieval
- interactions between client and server computers and
applications
- see definition of "protocol" above
- ISO
23950
- Z39.50
Maintenance Agency
|
