# coyote: NCDF_DATA__DEFINE

- coyote/FileIO
- [Source code]

NAME

      NCDF_DATA__DEFINE

PURPOSE

      This program is designed to make it easier to browse and read the
      data and metadata in netCDF and HDF files. The user can browse files,
      and read the data and metadata into main-level IDL variables. New netCDF
      and HDF files can be opened at any time. The user interacts with the
      program via a browser window (GUI) or directly through the methods of
      the object. The program implements an IDL object.
      Note that only HDF files with scientific datasets (SD) can be read currently.
      There is no support for VDATA objects or other objects sometimes found in HDF
      files. Also note that when variables are returned from HDF files, they are returned
      in a calibrated form, if calibration information about the variable is present in the
      file. Calibration information is presented as an extra variable attribute in the
      browser.
         calibratedData = calData.cal * (uncalibratedData - calData.offset)

AUTHOR

      FANNING SOFTWARE CONSULTING
      David Fanning, Ph.D.
      1645 Sheely Drive
      Fort Collins, CO 80526 USA
      Phone: 970-221-0438
      E-mail: david@idlcoyote.com
      Coyote's Guide to IDL Programming: http://www.idlcoyote.com

CATEGORY

      File I/O

CALLING SEQUENCE

      IDL> nCDFObject = Obj_New('NCDF_DATA', filename)

ARGUMENTS

      filename: The name of a netCDF or HDF file to open and browse.

KEYWORD PARAMETERS

      BROWSE:   If this keyword is set, the Browse Window is invoked as soon
                as the object is initiated.
      DESTROY_FROM_BROWSER:  As with all objects, this object is persistent until
                 it is destroyed. However, with this keyword set, the object will
                 be destroyed when the user closes the Browse Window.
      EXTENSION: In general, netCDF and HDF files use *.nc, *.ncf, *.ncdf of *.hdf file extensions to
                 identify themselves as netCDF or HDF files. Some users have their own file extensions.
                 You can use this keyword to identify the file extension you wish to use. If
                 set here, it will be used as the file filter in place of the normal file
                 extensions in DIALOG_PICKFILE.
                     obj = ('NCDF_DATA', file, EXTENSION='*.bin')
      NO_READ_ON_PARSE: Normally, when a file is opened it is parsed for information.
                 One piece of information is the minimum and maximum values of the variables.
                 This requires actually reading the variables. This can slow things down
                 considerably is the variable is large. Setting this keyword will suppress
                 the reading of the variables during the parsing of the data file, with the
                 result that no minimum or maximum values will be reported.

NOTES

      This program is designed to be flexible in how it is used, so it
      can be used in both interactive and non-interactive (called directly)
      ways. A less flexible way of interacting with the program is via the
      NCDF_BROWSER program, which is a front-end to this object.
      The netCDF and HDF file formats are thought to be "standards". And to
      a large extent, they are. But files are not always created to standards,
      and both netCDF and HDF files can be quirky. If you look carefully at the
      code you will see places where I work around quirks in the files I typically
      use on a daily basis. If you find you can't read a particular file, let me know
      about it. I may be able to improve the program in such as way that it can be read.
      This program is not meant to be the be-all and end-all of programs. Rather, it is
      a tool I use, and improve upon whenever necessary, in my own work with netCDF and HDF
      files. It will get better for all of us if you report problems to me directly.

METHODS

      The following methods can be used directly.
      ncdfObject -> Browse                             ; Use GUI to browse file data and metadata.
      ncdfObject -> OpenFile, filename                 ; Opens a new netCDF or HDF file.
      globalAttr = ncdfObject -> ReadGlobalAttr()      ; Return a structure containing global attributes.
      attribute = ncdfObject -> ReadAttribute(attrname); Return an attribute, identified by name.
      dim = ncdfObject -> ReadDimension(dimName)        ; Return a dimension, identified by name.
      variable = ncdfObject -> ReadVariable(varname)   ; Return a variable, identified by name.
      varstruct = ncdfObject -> ReadVariableWithAttr(varname)   ; Return a variable, identified by
                                                                ; name, along with its attributes.
      allData = ncdfObject -> ReadFile(filename)        ; Read all the data in the file, into structures.

EXAMPLE

      IDL> filename = 'example.nc'
      IDL> ncdfObj = Obj_New('NCDF_DATA', filename)
      IDL> ncdfObj -> Browse
      IDL> Obj_Destroy, ncdfObj

MODIFICATION HISTORY

      Written by:  David W. Fanning, 03 Feb 2008. Used ideas from many
          people, including Chris Torrence, Ken Bowman, Liam Gumely,
          Andrew Slater, and Paul van Delst.
      Added EXTENSION keyword, resizeable TLB, and ability to download
          individual global attibutes. DWF. 04 Feb 2008.
      Added ReadDimension and ReadVariableWithAttr methods. DWF. 05 Feb 2008.
      Ill-formed attribute names giving me fits. Now doing checks with IDL_VALIDNAME
           before creating structures. 06 February 2008. DWF.
      Same problem. Wide use of IDL_VALIDNAME everywhere it seems wise. 06 Feb 2008. DWF.
      Added functionality to read a variable with its attributes from the browser interface,
           and fixed a problem with reading CHAR values. 2 March 2008. DWF.
      Fixed a problem with changing variable name when reading variable plus attributes. 6 March 2008. DWF.
      Fixed a problem with not setting GLOBAL keyword when inquiring about global attribute. 6 March 2008. DWF.
      Made sure file was parsed before attempting to read variables and attributes to avoid errors. 7 March 2008. DWF.
      Small bug with variable attributes fixed. 18 Dec 2008. DWF.
      Added ability to read HDF files containing Scientific Datasets (SD). 21 February 2009. DWF.
      Added error handling and protection for NCDF variables that have a dimension of length zero. 22 April 2009. DWF.
      Added NO_READ_ON_PARSE keyword. 22 April 2009. DWF.
      Now convert NCDF CHAR type variables to strings on output. 22 April 2009. DWF
      Fixed a problem with the directory being correct when file name passed in. 11 May 2009. DWF.
      Added COUNT, OFFSET, and STRIDE keywords to ReadVariable method. 25 June 2009. DWF.
      When reading a netCDF variable by itself (without it's attributes), the program now looks for
         a SCALE_FACTOR and ADD_OFFSET attribute, and if found will apply this to the variable before
         it is returned to the user. 24 August 2009. DWF.
      Added the methods GetAttrNames, GetVarNames, GetVarAttrNames, and ReadVarAttr to retrieve specfic
         information from the data files. 16 November 2009. DWF.
      Modified the ReadVariableWithAttr method to include the number of dimensions (in the NDIMS field,
         and the dimensions (in the DIMS field) in the return structure. For HDF files, the DIMS field
         is a vector of the dimensions of the variable. For netCDF files, the DIMS field is a vector
         of dimension IDs for the dimensions of the variable. 27 Nov 2009. DWF.
      Andy Meigs alerted me to a problem creating a structure when the ncdf variable name
         is ill-formed according to IDL structure tag name rules. Fixed in the ReadFile method.
         30 November 2009. DWF.
      Added NO_NEW_FILE keyword to the BROWSE method. This keyword will suppress the OPEN FILE
         button on the browse interface. 3 Feb 2010. DWF.
      Made the default browser size a bit larger to accomodate longer variable names. 3 Feb 2010. DWF.
      Add a check for HDF/netCDF file type in the INIT method to better accommodate reading data
         from the file without first parsing the file. 16 March 2010. DWF.
      Changed the ReadVariable for netCDF files to now check for missing data, using either the
          depreciated missing_value attribute or the compliant _FillValue attribute. Missing data
          is now identified via new output keywords MISSINGINDICES and FILLVALUE, and missing data
          is not scaled or offset, if these operations are applied to the data prior to return.
          21 March 2010. DWF. Problem with these changes, fixed 23 March 2010. DWF.
      Fixed a problem with memory leakage when the input file cannot be read. 1 May 2010. DWF.
      Fixed a problem with memory leakage from created structures. 1 May 2010. DWF.
      Have done some work on parsing HDF-EOS swath files, but currently unused in code. 15 May 2010. DWF.
      Modified the ReadVariable method to check for 0 length dimensions when reading variables
          from HDF files. 21 July 2010. DWF.
      Modified the global attribute structure so that the "filename" field, which holds the
          name of the netCDF of HDF file is now named "ncdf_filename" or "hdf_filename". This
          will avoid conflicts with global attributes with "filename". 20 January 2011. DWF.
      Typo in the section reading calibration data fixed. 12 March 2013. DWF.
      Problem reading a variable containing no attributes fixed. 20 June 2014. DWF.