NEO Design and Requirements

Protocol and Interface Requirements

Status Server API
FITS header API
FITS pixel data API
Messaging API
External Messaging API

Component functionality Requirements

Status Server API Requirements

Purpose

The primary purpose of the status server API is to provide a method for sharing variables between clients, distributed over a local network. - Status -> Primarily Current telescope and instrument configuration parameters. - Only the ones that need to be shared. "public" - Expand on variables - Limited types/conversions? - Low to moderate update... N Hz max frequency ... Examples of what is reasonable and what's not. - Small size Examples of reasonable. - The main purpose of the API will be to provide function call-based access to these variables in the C language. Clients acting as "providers" use API functions to "create" entries and "put" new values, while clients acting as "consumers" use API functions to "get" or "monitor" these same variables. These are the main functions provided by the API. - No history/archive - Security - clients can count on having their entries preserved (because server serializes periodically) but does not provide the type of "on-disk" security of a commercial database for each update. - Philosophy - Filesystem model - Simple to use, even for non-professional programmers. - Restrict to doing things that are otherwise difficult or inefficient to implement elsewhere. Efficiency -> programming effort, network bandwidth. - Client design philosophy: Clients should be able to handle a completely empty status server upon startup. While it is a reasonable use of the status server to store configuration parameters to be re-read on startup (such as those typically stored in a .ini or .rc file), each client must have a graceful fallback for dealing with missing variables. A builtin template should exist within all providers which can regenerate all variables it provides.

Terminology

Server

Refers collectively to the data store in which status values lives. Even if it is not implemented as a single program, there will only be one hierarchy visible to a client on its local network.

Client

Any program which uses this API to submit and/or request status information. A client can be both a Provider and a Consumer.

Provider

A Client, whenever it creates or updates an entry in the status server.

Consumer

A Client, whenever it requests an entry, or sets a monitor on an entry.

Name/Value Pairs

What the status server stores.

Aside: When it comes to the design of our status server, we may provide a way for a client to get a name who's value is the result of some action taken by the status server. Even though this is a function/return model, it will be disguised as a name/value lookup to the client.

Names

The hierarchical keys used by clients to look up status items.

Values

The primary field of interest associated with a name. Other related fields may be stored with the value in the status server. Here are some proposed possibilities:

• Flags - Indicating if the value is valid, out of range, uninitialized, etc.
• Default type - The type (integer, float, string) with which this item was first created.
• Enforced type - Should the status server accept a value update if it doesn't convert "cleanly" to its default type or should it simply set the default-type version of its value to "invalid"?
• Normal range - Max/min limits for integer and floating point types.
• Enforced range - Should the status server accept a value update if it is outside the allowable range or should it be responsible for returning an error if someone tries to set an illegal value.
• Enforced selected values - This is the same thing as having "enum" in addition to integer, float, and string types.
• Expiration - If a client gets a value after the expiration time, or if it had a monitor set when a value expired, it should be told that the value is now invalid.
• Timestamp - The time at which the value was submitted to the status server.
• Lifetime - (See Open/Touch/Create below also). If set when the variable is created (or occaisionally reconfigured but remains "sticky", then each time a client puts a new value into the server, the server should take care of setting Expiration = Timestamp + Lifetime. Alternatively, lifetime could be something set on a per-put-call basis, but that makes the API look somewhat more complex for clients that don't care about this lifetime feature.

Stat/Exists/Contains

The Consumer action of checking for the existance of a Name in the status server.

Open/Touch/Create

We need some word that describes the Provider action of making sure a Name/Value Pair exists in the status server.

Put

The preferred name for the Provider action of updating a value in the status server.

Get

The preferred name for the Consumer action of an immediate request for a value.

Monitor

The preferred name for the Consumer action of requesting continuing interest in a value. The following options may control the behavior of a monitor:

Deadband

Applies to integer and floating point types. Instruct the server to suppress an update if the previous value sent to this client is within "deadband" units of the current value.

Age

Applies to all types. Instruct the server to suppress an update if the previous value sent to this client is less than "age" time-units old. Why age? Well, we're not completely happy with the name, but it basically implies the server will always allow the last update to age at least the specified time before sending a new update.

(Aside: this implies that the server must keep a close watch on the age of all monitors, since the event of the age of the last sent value reaching the threshold could trigger an update, even if the current value didn't happen to change at that time. Consider the following sequence of events:

• t=1 Provider puts value N = 7.0 (server saves timestamp)
• t=2 Client monitors N with age = 5 seconds.
• t=2 Server tells client "N is 7.0" (already has age of 1 second)
• t=4 Provider puts value N = 7.5 (no message to client, age is 3 seconds)
• t=6 Server has to wake up to tell client "N is now 7.5" (age 2 seconds)

If both age and deadband options are used, the server will suppress an update if either are still within the specified limits. In other words, age sets a maximum refresh rate, and deadband selects a range of values which the client considers equivalent.

Value Types

• ASCII strings [maxlen]
• Integers [bits]
• Floats [bits]
• Arrays [maxlen, max dim]
• Warning limits
• Conversion
• Links

Provider Requirements

• Push mode
• Pull from provider by server
  1. Link to some C code/routine
  2. Proxy for another server
• Lifetime

Consumer requirements

• Pull
• Listen
• Deadband/Tolerance
• Age
• Type conversion

Breakdown of Client API Calls

call	inputs	outputs	Who?	Nonblocking?	purpose
logon	none? client id?	PASS/FAIL	Clients	N	Get the connect with server phase out of the way and make sure the server is up. Alternatively, or in addition, all other API calls should first check that the connection is up.
logoff	none	PASS/FAIL	Clients	N	Free resources and shut down connection.
touch	name, type? min? max? lifetime?	PASS/FAIL	Provider	N	Three uses (maybe want three functions?): (1) Create variable if it doesn't exist (2) Register interest in having write permission (3) Reconfigure extra fields like lifetime, min, max, etc.
put	name, value lifetime?	PASS/FAIL	Provider	N	Put a new value.
stat	name	PASS/FAIL	Consumer	N	Test if a variable exists.
get	name	PASS/FAIL, value	Consumer	N	Get a value.
monitor	name, deadband? age?	PASSFAIL	Consumer	Y	Set up a monitor
kill_monitor	name	PASS/FAIL	Consumer	Y	Stop monitor

FITS header API Requirements

• Distributed
• Concurrent updates
• Adjusts header block size dynamically
• No need for templates
• Support for building multiple header blocks (MEF)
• Dynamic sorting mechanism (index numbers)
• Error/integrity checking
  • valid registered names?
  • Headers that CADC requires: are they present and valid?
  • Headers that CADC uses if present: are they valid?
• Handle missing/stale data (e.g. RUNID) by printing warning in engineering mode, or by failing in observing mode.

FITS pixel data API Requirements

• Distributed
• Concurrent data streams from multiple chips.
• Pass scrambled data to capture agent?
• Way to specify chips and rasters.

Messaging API Requirements

• Command string assembly and argument parsing.
• Command passing interface (cli_cmd())
• Response (PASS/FAIL return from cli_cmd())
• More complex error/completion status? (get from status server?)

External Messaging API Requirements

• Same as requirements for internal messaging
• Anything special needed by Queue?

Pipeline/Archive Interface Requirements

• Way to register interest in any new exposures when completed.
• Limited by instrument?
• Limited by exposure type?
• Must not be able to hang the system!

Image Viewer Requirements

• Standalone, easy to install binary, portable to common platforms.
• Documented and easy to use?
• Efficient handling of large data sets.
• Efficient handling of data on machines requiring byte-swapping.
• Support for Pseudocolor displays (8-bit colortable)
• Support for Truecolor displays (16? 24? 32?)
• Builtin support for MEF display
• View FITS header list
• Manipulate FITS headers? (or do with separate tool)
• Color/scale adjustment controls
  • Scale adjustment without losing resolution
  • Incremental or relative color adjustment (fine tuning?)
  • Histogram equalization
  • Iraf Zimage
• Visualization features
  • Grapher, x-cuts, y-cuts, arbitrary cuts (rotate image?)
  • Pixel Table
  • Zoom window
  • Panner window
  • Progressive real-time display or ability to update region?
    Suggestion from dennis: just show one step at 25% and a final draw.
  • 3-color composite image
  • Subsampled image display
  • Averaged/binned image display
• World Coording System support
• Catalog interface
• Data server
• Image overlays
• Image analysis
  • FWHM
  • PSF
  • Aperture photometry
  • Pass data to external tool? Or...
• Easily extensible (plug-ins?)
• Remote control

TCS Agent Requirements

• Fake mode (simulate the hardware)
• Upper limit timeouts on "slow" commands
• Interruptable (SIGINT) on some "slow" commands (others print "sorry" message.)
• Commands from NEO
  • (tcsh) Grab current status
  • (tcsh) Move guide probe
  • (tcsh) ... Look in tcsh for others....
  • (telofset) Do small offsets
  • Request new target
  • Wait for guider
• Status from TCS to NEO
  • telescope data like available now with tcsh (Look in tcsh.par)
  • more, higher level status, like ready/not ready, etc.
  • guider image and guiding quality info
• Status from NEO to TCS
  • obs run ID & observer info (1/night or 1/run change)
  • instrument & detector stuff (1/night)
  • exposure info (1/exposure)
    • exposure type, length, etc.?
    • shutter status
    • More...? Look in 12kcom.par

Data Capture Agent Requirements

• Concurrent exposures?

Status Server Protocol Requirements

• Support for value types in the API
• Support for arbitrary future value types?
• Support to meet any provider and consumer's requirements
• Provide open diagnostic capability with minimum tools

Status Server Requirements

• Support for value types in the API
• Redundancy (Backups - Sybase replication server?)
• Ability to point to "fake" server on another machine for testing
• Avoid duplication
• Atomicity/Locking/Synchronization
• Deterministic - Listen can't be 5 min?
• Data integrity on disk
• Performance limits (times/sec lookup times, make use of SMP?)
• Portability, maintainability, configurability (Cross platform and network dependencies)

Status Server Selection

• Sybase (running on NT/Linux)
• EPICS channel access with a realtime system
• Gemini Status Server: Channel Access, IMP (DAO)
• Our own status server based on the EPICS portable channel access server
• Our own status server based on Sidik's socket repeater program
• Our own status server based on inhouse/vendor supplied, distributed tools such as GNU AVL trees/Sybase/EPICS CA/Inhouse libraries. (most flexible).

Command Sequencer Requirements

• Multiple "progress:" messages (one from each agent?)