README for DB-All.e Python bindings

The DB-All.e Python bindings provide 2 levels of access to a DB-All.e database: a complete API similar to the Fortran and C++ API, and a high-level API called volnd that allows to automatically export matrices of data out of the database.

Contents

The DB-All.e API

The 'dballe' module has a few global methods:

dballe.describe_level(ltype1: int, l1: int=None, ltype2: int=None, l2: int=None) -> str
Return a string description for a level
dballe.describe_trange(pind: int, p1: int=None, p2: int=None) -> str
Return a string description for a time range
dballe.var(code, val: Any=None) -> dballe.Var
Query the DB-All.e variable table returning a Var, optionally initialized with a value
dballe.varinfo(str) -> str
Query the DB-All.e variable table returning a Varinfo

and several classes, documented in their own sections.

dballe.Var

Var holds a measured value, which can be integer, float or string, and a dballe.Varinfo with all available information (description, unit, precision, ...) related to it.

Var objects can be created from a dballe.Varinfo object, and an optional value. Omitting the value creates an unset variable.

Examples:

v = dballe.Var(table["B12101"], 32.5)
# v.info returns detailed informations about the variable in a Varinfo object.
print("%s: %s %s %s" % (v.code, str(v), v.info.unit, v.info.desc))

Members

Var.code
variable code
Var.info
Varinfo for this variable
Var.isset
true if the value is set
Var.enq() -> Union[str, float, int]
get the value of the variable, as int, float or str according the variable definition
Var.enqa(code: str) -> Optional[dballe.Var]
get the variable for the attribute with the given code, or None if not found
Var.enqc() -> str
get the value of the variable, as a str
Var.enqd() -> float
get the value of the variable, as a float
Var.enqi() -> int
get the value of the variable, as an int
Var.format(default: str=) -> str
return a string with the formatted value of the variable
Var.get(default: Any=None) -> Union[str, float, long, Any]
get the value of the variable, with a default if it is unset
Var.get_attrs() -> List[dballe.Var]
get the attributes of this variable
Var.seta(var: dballe.Var)
set an attribute in the variable
Var.unseta(code: str)
unset the given attribute from the variable

dballe.Varinfo

Varinfo object holds all possible information about a variable, such as its measurement unit, description and number of significant digits.

Varinfo objects cannot be instantiated directly, and are created by querying dballe.Vartable objects.

Members

Varinfo.bit_len
number of bits used to encode the value in BUFR
Varinfo.bit_ref
reference value added after scaling, for BUFR decoding
Varinfo.code
variable code
Varinfo.desc
description
Varinfo.len
number of significant digits
Varinfo.scale
scale of the value as a power of 10
Varinfo.type
return a string describing the type of the variable (string, binary, integer, decimal)
Varinfo.unit
measurement unit

dballe.Vartable

Collection of Varinfo objects indexed by WMO BUFR/CREX table B code.

A Vartable is instantiated by the name (without extension) of the table file installed in wreport's data directory (normally, /usr/share/wreport/):

table = dballe.Vartable("B0000000000000023000")
print(table["B12101"].desc)

for i in table:
    print(i.code, i.desc)

Members

Vartable.pathname
name of the table
Vartable.get_bufr(basename: str=None, originating_centre: int=0, originating_subcentre: int=0,master_table_number: int=0, master_table_version_number: int=None, master_table_version_number_local: int=0) -> dballe.Vartable

Look up a table B file using the information given, then load BUFR information from it.

You need to provide either basename or master_table_version_number.

Vartable.get_crex(basename: str=None, edition_number=2, originating_centre: int=0, originating_subcentre: int=0,master_table_number: int=0, master_table_version_number: int=None,master_table_version_number_bufr: int=None, master_table_version_number_local: int=0) -> dballe.Vartable

Look up a table B file using the information given, then load CREX information from it.

You need to provide either basename or master_table_version_number or master_table_version_number_bufr.

Vartable.load_bufr(pathname: str) -> dballe.Vartable
Load BUFR information from a Table B file and return it as a dballe.Vartable.
Vartable.load_crex(pathname: str) -> dballe.Vartable
Load CREX information from a Table B file and return it as a dballe.Vartable.

dballe.Level

Level or layer.

Constructor: Level(ltype1: int=None, l1: int=None, ltype2: int=None, l2: int=None)

Members

Level.l1
value of the level or of the first layer
Level.l2
value of the second layer
Level.ltype1
type of the level or of the first layer
Level.ltype2
type of the second layer

dballe.Trange

Time range.

Constructor: Trange(pind: int=None, p1: int=None, p2: int=None)

Members

Trange.p1
Time range P1 indicator
Trange.p2
Time range P2 indicator
Trange.pind
Time range type indicator

dballe.Station

Station information.

Constructor: Station(report: str, lat: float, lon: float, ident: str=None)

Members

Station.ident
mobile station identifier
Station.lat
station latitude
Station.lon
station longitude
Station.report
report for this station

dballe.DBStation

Station information with database ID.

Constructor: Station(report: str, id: int, lat: float, lon: float, ident: str=None)

Members

DBStation.id
database ID for this station
DBStation.ident
mobile station identifier
DBStation.lat
station latitude
DBStation.lon
station longitude
DBStation.report
report for this station

dballe.Data

key-value representation of a value with its associated metadata

dballe.BinaryMessage

Binary message.

This is basically a simple wrapper around a bytes() object, providing extra information about the filename, offset and index where the message data was read. Is it used by dballe.File to return the binary messages it reads.

Members

BinaryMessage.encoding
message encoding
BinaryMessage.index
index of the message in the input file, or None if unknown
BinaryMessage.offset
offset of the message in the input file, or None if unknown
BinaryMessage.pathname
pathname of the file the message came from, or None if unknown

dballe.File

Read-only access to files with weather bulletins in BUFR or CREX format.

No write functions are supported: to write files, you can simply write dballe.BinaryMessage objects or encoded messages to normal Python files.

Constructor: File(file: Union[str, File], encoding: str=None)

file can be a file name, or a file-like object. If a file-like object supports fileno(), that file descriptor is dup()-ed and used for efficient reading. Otherwise, file.read() is called to load the data to read in memory.

If encoding is omitted, it is auto detected by looking at the first byte of the file only. Files with leading padding data will not be detected properly, and you need to explicitly specify the encoding to read them.

Example usage:

with dballe.File("test.bufr", "BUFR") as f:
    for binmsg in f:
        print("#{m.index}: {m.pathname}:{m.offset}: {m.encoding} message".format(m=binmsg))

Members

File.encoding
get the file encoding
File.name
get the file name

dballe.Message

The contents of a decoded BUFR or CREX message.

DB-All.e can interpret the contents of most weather messages commonly in use, and represent them as variables identified by dballe.Level, dballe.Trange, datetime, coordinates, network, and mobile station identifier.

A message contains only one reference station (coordinates, network, mobile station identifier), only one reference datetime, and many (level, trange, varcode, value) variables.

Variables that describe the station are accessible using None for level and trange.

Constructor: Message(type: str)

type is a string identifying the message type, and it will affect how the message will be encoded by the exporter.

Available values are:
  • generic
  • synop
  • pilot
  • temp
  • temp_ship;
  • airep
  • amdar
  • acars
  • ship
  • buoy
  • metar
  • sat

Example usage:

importer = dballe.Importer("BUFR")
with importer.from_file("test.bufr") as f:
    for msg in f:
        print("{m.report},{m.coords},{m.ident},{m.datetime},{m.type}".format(m=msg))

Members

Message.coords
message coordinates
Message.datetime
message datetime
Message.ident
message mobile station identifier
Message.report
message report
Message.type
message type
Message.get(level: dballe.Level, trange: dballe.Trange, code: str) -> Union[dballe.Var, None]
Get a Var given its level, timerange, and varcode; returns None if not found
Message.get_named(name: str) -> Union[dballe.Var, None]
Get a Var given its shortcut name; returns None if not found
Message.query_data(query: Dict[str, Any]) -> dballe.CursorData
Query the variables in the message
Message.query_station_data(query: Dict[str, Any]) -> dballe.CursorStationData
Query the station variables in the message
Message.query_stations(query: Dict[str, Any]) -> dballe.CursorStation
Query the station data in the message
Message.set(level: dballe.Level, trange: dballe.Trange, var: dballe.Var)
Set a Var given level and timerange
Message.set_named(name: str, var: dballe.Var)
Set a Var given its shortcut name

dballe.Importer

Message importer.

This is the engine that decodes binary messages and interprets their contents using a uniform data model.

Note that one binary message is often decoded to multiple data messages, in case, for example, of compressed BUFR files.

Constructor: Importer(encoding: str, simplified: bool=True)

encoding can be "BUFR" or "CREX".

simplified control whether messages are constructed using standard levels and time ranges, or using the exact levels and time ranges contained in the input. For example, a simplified intepretation of a synop message will place the temperature at 2M above ground, regardless of the reported sensor height. A non-simplified import will place the temperature reading at the reported sensor height.

When a message is imported in simplified mode, the actual context information will be stored as data attributes.

Example usage:

importer = dballe.Importer("BUFR")
with importer.from_file("test.bufr") as f:
    for msg in f:
        print("{m.report},{m.coords},{m.ident},{m.datetime},{m.type}".format(m=msg))

importer = dballe.Importer("BUFR")
with dbale.File("test.bufr") as f:
    for binmsg in f:
        msgs = importer.from_binary(binmsg)
        for msg in msgs:
            print("#{b.index}: {m.report},{m.coords},{m.ident},{m.datetime},{m.type}".format(b=binmsg, m=msg))

Members

Importer.from_binary(binmsg: dballe.BinaryMessage) -> Sequence[dballe.BinaryMessage]
Decode a BinaryMessage to a tuple of dballe.Message objects
Importer.from_file(file: Union[dballe.File, str, File]) -> dballe.ImporterFile

Wrap a dballe.File into a sequence of tuples of dballe.Message objects.

file can be a dballe.File, a file name, or a file-like object. A dballe.File is automatically constructed if needed, using the importer encoding.

dballe.ImporterFile

Message importer iterating over the contents of a a dballe.File.

This is never instantiated explicitly, but is returned by Importer.from_file().

It can be used in a context manager, and it is an iterable that yields tuples of dballe.Message objects.

dballe.Exporter

Message exporter.

This is the engine that can reconstruct a standard BUFR or CREX message from the contents of a dballe.Message.

Members

Exporter.to_binary(contents: Union[dballe.Message, Sequence[dballe.Message], Iterable[dballe.Message]]) -> bytes
Encode a dballe.Message or a sequence of dballe.Message into a bytes object.

dballe.DB

DB-All.e database access.

Many methods are the same in dballe.DB and dballe.Transaction. The versions in dballe.DB are implemented by automatically creating a temporary transaction and running the equivalent dballe.Transaction method inside it.

dballe.DB objects are not constructed explicitly, but via one of the DB.connect or DB.connect_test class methods.

Examples:

# Connect to a database and run a query
db = dballe.DB.connect_from_file("db.sqlite")
query = {latmin=44.0, latmax=45.0, lonmin=11.0, lonmax=12.0}

# The result is a dballe.Cursor (dballe.CursorData in this case), which can
# be iterated to get results as dict objects.
for row in db.query_data(query):
    print(row["lat"], row["lon"], row["var"], row.var().format("undefined"))

# Insert 2 new variables in the database
db.insert_data({
    "lat": 44.5, "lon": 11.4,
    "level": dballe.Level(1),
    "trange": dballe.Trange(254),
    "date": datetime.datetime(2013, 4, 25, 12, 0, 0),
    "B11101": 22.4,
    "B12103": 17.2,
})

Members

DB.attr_insert_data(varid: int, attrs: Dict[str, Any])
Insert new attributes into the database
DB.attr_insert_station(varid: int, attrs: Dict[str, Any])
Insert new attributes into the database
DB.attr_query_data(varid: int) -> Dict[str, Any]
query data attributes
DB.attr_query_station(varid: int) -> Dict[str, Any]
query station data attributes
DB.attr_remove_data(varid: int, attrs: Iterable[str])
Remove attributes from data variables
DB.attr_remove_station(varid: int, attrs: Iterable[str])
Remove attributes from station variables
DB.connect(url: str) -> dballe.DB
create a DB to access a database identified by a DB-All.e URL
DB.connect_from_file(name: str) -> dballe.DB
create a DB to access a SQLite file
DB.connect_from_url(url: str) -> dballe.DB
create a DB to access a database identified by a DB-All.e URL (deprecated, use connect instead)
DB.connect_test() -> dballe.DB
Create a DB for running the test suite, as configured in the test environment
DB.disappear()
Remove all DB-All.e tables and data from the database, if possible
DB.get_default_format() -> str
get the default DB format
DB.import_messages(messages: Union[dballe.Message, Sequence[dballe.Message], Iterable[dballe.Message], dballe.ImporterFile], report: str=None, import_attributres: bool=False, update_station: bool=False, overwrite: bool=False)

Import one or more Messages into the database.

messages can be:

report is the network name to use for importing the data. If left to None, the network is selected automatically from the message type

import_attributes if set to True, requests the variable attributes to also be imported.

update_station, if set to True, station information is merged with existing one in the database. If false (default), station information is imported only when the station did not exist in the database.

overwrite, if set to True, causes existing information already in the database to be overwritten. If false (default), trying to import a message which contains data already present in the database causes the import to fail.

varlist, if set to a string in the same format as the varlist query parameter, only imports data whose varcode is in the list.

DB.insert_data(record: Union[Dict[str, Any], dballe.Cursor], can_replace: bool=False, can_add_stations: bool=False) -> Dict[str, int]

Insert data values in the database

The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.

DB.insert_station_data(record: Union[Dict[str, Any], dballe.Cursor], can_replace: bool=False, can_add_stations: bool=False) -> Dict[str, int]

Insert station values in the database

The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.

DB.is_url(url: str) -> bool
Checks if a string looks like a DB-All.e DB url
DB.query_data(query: Dict[str, Any]) -> dballe.CursorData
Query the variables in the database
DB.query_messages(query: Dict[str, Any]) -> dballe.CursorMessage
Query the database returning the matching data as Message objects
DB.query_station_data(query: Dict[str, Any]) -> dballe.CursorStationData
Query the station variables in the database
DB.query_stations(query: Dict[str, Any]) -> dballe.CursorStation
Query the station archive in the database
DB.query_summary(query: Dict[str, Any]) -> dballe.CursorSummary
Query the summary of the results of a query
DB.remove(query: Dict[str, Any])
Remove data variables from the database (deprecated)
DB.remove_all()
Remove all data from the database
DB.remove_data(query: Dict[str, Any])
Remove data variables from the database
DB.remove_station_data(query: Dict[str, Any])
Remove station variables from the database
DB.reset(repinfo_file: str=None)
Reset the database, removing all existing Db-All.e tables and re-creating them empty.
DB.set_default_format(format: str)
set the default DB format
DB.transaction(readonly: bool=False) -> dballe.Transaction
Create a new database transaction
DB.vacuum()
Perform database cleanup operations

dballe.Transaction

DB-All.e transaction

A Transaction is used to execute DB operations in an all-or-nothing fashion. In fact, most DB methods are implemented using a short-lived temporary transaction.

You cannot have more than one active dballe.Transaction for each dballe.DB. An attempt to start a second one will result in an exception being raised. Note that dballe.DB functions like insert_data or export_to_file create a temporary transaction to run, and so they will also fail if a transaction is currently open. The general idea is that all database work should be done inside a transaction.

Transactions run using the REPEATABLE READ isolation level of the underlying database. This usually means that modifications performed inside a transaction are not visible to other database connections until the transaction is committed. If a transaction is rolled back, all changes done with it are undone.

Transactions can also be used as context managers, like this:

with db.transaction() as t:
    for i in range(10):
        t.insert({
            "lat": 44.5 + i, "lon": 11.4 + i, "level": (1,),
            "trange": (254,), "date": datetime.datetime(2013, 4, 25, 12, 0, 0),
            "B11101": 22.4 + i, "B12103": 17.2
        })

The dballe.Transaction methods are the same as those in dballe.DB. The version in dballe.DB is implemented by automatically creating a temporary transaction and running the dballe.Transaction method inside it.

Members

Transaction.attr_insert_data(varid: int, attrs: Dict[str, Any])
Insert new attributes into the database
Transaction.attr_insert_station(varid: int, attrs: Dict[str, Any])
Insert new attributes into the database
Transaction.attr_query_data(varid: int) -> Dict[str, Any]
query data attributes
Transaction.attr_query_station(varid: int) -> Dict[str, Any]
query station data attributes
Transaction.attr_remove_data(varid: int, attrs: Iterable[str])
Remove attributes from data variables
Transaction.attr_remove_station(varid: int, attrs: Iterable[str])
Remove attributes from station variables
Transaction.commit()
commit the transaction
Transaction.import_messages(messages: Union[dballe.Message, Sequence[dballe.Message], Iterable[dballe.Message], dballe.ImporterFile], report: str=None, import_attributres: bool=False, update_station: bool=False, overwrite: bool=False)

Import one or more Messages into the database.

messages can be:

report is the network name to use for importing the data. If left to None, the network is selected automatically from the message type

import_attributes if set to True, requests the variable attributes to also be imported.

update_station, if set to True, station information is merged with existing one in the database. If false (default), station information is imported only when the station did not exist in the database.

overwrite, if set to True, causes existing information already in the database to be overwritten. If false (default), trying to import a message which contains data already present in the database causes the import to fail.

varlist, if set to a string in the same format as the varlist query parameter, only imports data whose varcode is in the list.

Transaction.insert_data(record: Union[Dict[str, Any], dballe.Cursor], can_replace: bool=False, can_add_stations: bool=False) -> Dict[str, int]

Insert data values in the database

The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.

Transaction.insert_station_data(record: Union[Dict[str, Any], dballe.Cursor], can_replace: bool=False, can_add_stations: bool=False) -> Dict[str, int]

Insert station values in the database

The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.

Transaction.query_data(query: Dict[str, Any]) -> dballe.CursorData
Query the variables in the database
Transaction.query_messages(query: Dict[str, Any]) -> dballe.CursorMessage
Query the database returning the matching data as Message objects
Transaction.query_station_data(query: Dict[str, Any]) -> dballe.CursorStationData
Query the station variables in the database
Transaction.query_stations(query: Dict[str, Any]) -> dballe.CursorStation
Query the station archive in the database
Transaction.query_summary(query: Dict[str, Any]) -> dballe.CursorSummary
Query the summary of the results of a query
Transaction.remove(query: Dict[str, Any])
Remove data variables from the database (deprecated)
Transaction.remove_all()
Remove all data from the database
Transaction.remove_data(query: Dict[str, Any])
Remove data variables from the database
Transaction.remove_station_data(query: Dict[str, Any])
Remove station variables from the database
Transaction.rollback()
roll back the transaction

dballe.CursorStation

A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.

A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.

For example:

with self.db.query_data({...}) as cur:
    for row in cur:
        print("Station:", cur["station"])
        print("Var:", cur["var"], cur.enqf(cur["var"]))

Members

CursorStation.query
return a dict with a query to select exactly the current value at this cursor
CursorStation.remaining
number of results still to be returned
CursorStation.enqd(key: str) -> Union[float, None]
Return the float value for a keyword
CursorStation.enqf(key: str) -> Union[str, None]
Return the formatted string value for a keyword
CursorStation.enqi(key: str) -> Union[int, None]
Return the integer value for a keyword
CursorStation.enqs(key: str) -> Union[str, None]
Return the string value for a keyword

dballe.CursorStationData

A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.

A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.

For example:

with self.db.query_data({...}) as cur:
    for row in cur:
        print("Station:", cur["station"])
        print("Var:", cur["var"], cur.enqf(cur["var"]))

Members

CursorStationData.data
return a dballe.Data which can be used to insert into a database the current cursor value
CursorStationData.data_dict
return a dict which can be used to insert into a database the current cursor value
CursorStationData.remaining
number of results still to be returned
CursorStationData.enqd(key: str) -> Union[float, None]
Return the float value for a keyword
CursorStationData.enqf(key: str) -> Union[str, None]
Return the formatted string value for a keyword
CursorStationData.enqi(key: str) -> Union[int, None]
Return the integer value for a keyword
CursorStationData.enqs(key: str) -> Union[str, None]
Return the string value for a keyword

dballe.CursorData

A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.

A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.

For example:

with self.db.query_data({...}) as cur:
    for row in cur:
        print("Station:", cur["station"])
        print("Var:", cur["var"], cur.enqf(cur["var"]))

Members

CursorData.data
return a dballe.Data which can be used to insert into a database the current cursor value
CursorData.data_dict
return a dict which can be used to insert into a database the current cursor value
CursorData.remaining
number of results still to be returned
CursorData.enqd(key: str) -> Union[float, None]
Return the float value for a keyword
CursorData.enqf(key: str) -> Union[str, None]
Return the formatted string value for a keyword
CursorData.enqi(key: str) -> Union[int, None]
Return the integer value for a keyword
CursorData.enqs(key: str) -> Union[str, None]
Return the string value for a keyword

dballe.CursorStationDB

A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.

A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.

For example:

with self.db.query_data({...}) as cur:
    for row in cur:
        print("Station:", cur["station"])
        print("Var:", cur["var"], cur.enqf(cur["var"]))

Members

CursorStationDB.query
return a dict with a query to select exactly the current value at this cursor
CursorStationDB.remaining
number of results still to be returned
CursorStationDB.enqd(key: str) -> Union[float, None]
Return the float value for a keyword
CursorStationDB.enqf(key: str) -> Union[str, None]
Return the formatted string value for a keyword
CursorStationDB.enqi(key: str) -> Union[int, None]
Return the integer value for a keyword
CursorStationDB.enqs(key: str) -> Union[str, None]
Return the string value for a keyword
CursorStationDB.remove()
Remove the data currently addressed by the cursor

dballe.CursorStationDataDB

A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.

A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.

For example:

with self.db.query_data({...}) as cur:
    for row in cur:
        print("Station:", cur["station"])
        print("Var:", cur["var"], cur.enqf(cur["var"]))

Members

CursorStationDataDB.data
return a dballe.Data which can be used to insert into a database the current cursor value
CursorStationDataDB.data_dict
return a dict which can be used to insert into a database the current cursor value
CursorStationDataDB.remaining
number of results still to be returned
CursorStationDataDB.enqd(key: str) -> Union[float, None]
Return the float value for a keyword
CursorStationDataDB.enqf(key: str) -> Union[str, None]
Return the formatted string value for a keyword
CursorStationDataDB.enqi(key: str) -> Union[int, None]
Return the integer value for a keyword
CursorStationDataDB.enqs(key: str) -> Union[str, None]
Return the string value for a keyword
CursorStationDataDB.insert_attrs(attrs: Dict[str, Any])
Insert or update attributes for the current variable
CursorStationDataDB.remove()
Remove the data currently addressed by the cursor
CursorStationDataDB.remove_attrs(attrs: Iterable[str])
Remove attributes from the current variable

dballe.CursorDataDB

A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.

A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.

For example:

with self.db.query_data({...}) as cur:
    for row in cur:
        print("Station:", cur["station"])
        print("Var:", cur["var"], cur.enqf(cur["var"]))

Members

CursorDataDB.data
return a dballe.Data which can be used to insert into a database the current cursor value
CursorDataDB.data_dict
return a dict which can be used to insert into a database the current cursor value
CursorDataDB.remaining
number of results still to be returned
CursorDataDB.enqd(key: str) -> Union[float, None]
Return the float value for a keyword
CursorDataDB.enqf(key: str) -> Union[str, None]
Return the formatted string value for a keyword
CursorDataDB.enqi(key: str) -> Union[int, None]
Return the integer value for a keyword
CursorDataDB.enqs(key: str) -> Union[str, None]
Return the string value for a keyword
CursorDataDB.insert_attrs(attrs: Dict[str, Any])
Insert or update attributes for the current variable
CursorDataDB.remove()
Remove the data currently addressed by the cursor
CursorDataDB.remove_attrs(attrs: Iterable[str])
Remove attributes from the current variable

dballe.CursorSummaryDB

A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.

A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.

For example:

with self.db.query_data({...}) as cur:
    for row in cur:
        print("Station:", cur["station"])
        print("Var:", cur["var"], cur.enqf(cur["var"]))

Members

CursorSummaryDB.query
return a dict with a query to select exactly the current value at this cursor
CursorSummaryDB.remaining
number of results still to be returned
CursorSummaryDB.enqd(key: str) -> Union[float, None]
Return the float value for a keyword
CursorSummaryDB.enqf(key: str) -> Union[str, None]
Return the formatted string value for a keyword
CursorSummaryDB.enqi(key: str) -> Union[int, None]
Return the integer value for a keyword
CursorSummaryDB.enqs(key: str) -> Union[str, None]
Return the string value for a keyword
CursorSummaryDB.remove()
Remove the data currently addressed by the cursor

dballe.CursorSummarySummary

A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.

A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.

For example:

with self.db.query_data({...}) as cur:
    for row in cur:
        print("Station:", cur["station"])
        print("Var:", cur["var"], cur.enqf(cur["var"]))

Members

CursorSummarySummary.query
return a dict with a query to select exactly the current value at this cursor
CursorSummarySummary.remaining
number of results still to be returned
CursorSummarySummary.enqd(key: str) -> Union[float, None]
Return the float value for a keyword
CursorSummarySummary.enqf(key: str) -> Union[str, None]
Return the formatted string value for a keyword
CursorSummarySummary.enqi(key: str) -> Union[int, None]
Return the integer value for a keyword
CursorSummarySummary.enqs(key: str) -> Union[str, None]
Return the string value for a keyword

dballe.CursorSummaryDBSummary

A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.

A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.

For example:

with self.db.query_data({...}) as cur:
    for row in cur:
        print("Station:", cur["station"])
        print("Var:", cur["var"], cur.enqf(cur["var"]))

Members

CursorSummaryDBSummary.query
return a dict with a query to select exactly the current value at this cursor
CursorSummaryDBSummary.remaining
number of results still to be returned
CursorSummaryDBSummary.enqd(key: str) -> Union[float, None]
Return the float value for a keyword
CursorSummaryDBSummary.enqf(key: str) -> Union[str, None]
Return the formatted string value for a keyword
CursorSummaryDBSummary.enqi(key: str) -> Union[int, None]
Return the integer value for a keyword
CursorSummaryDBSummary.enqs(key: str) -> Union[str, None]
Return the string value for a keyword

dballe.CursorMessage

A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.

A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.

For example:

with self.db.query_data({...}) as cur:
    for row in cur:
        print("Station:", cur["station"])
        print("Var:", cur["var"], cur.enqf(cur["var"]))

Members

CursorMessage.message
dballe.Message object with the current message
CursorMessage.query
return a dict with a query to select exactly the current value at this cursor
CursorMessage.remaining
number of results still to be returned
CursorMessage.enqd(key: str) -> Union[float, None]
Return the float value for a keyword
CursorMessage.enqf(key: str) -> Union[str, None]
Return the formatted string value for a keyword
CursorMessage.enqi(key: str) -> Union[int, None]
Return the integer value for a keyword
CursorMessage.enqs(key: str) -> Union[str, None]
Return the string value for a keyword

dballe.Explorer

Browser for a summary of DB-All-e database of message contents

Members

Explorer.all_levels
get all level values
Explorer.all_reports
get all report values
Explorer.all_stations
get all stations
Explorer.all_stats
get the stats for all values
Explorer.all_tranges
get all time range values
Explorer.all_varcodes
get all varcode values
Explorer.levels
get all the level values currently selected
Explorer.reports
get all the report values currently selected
Explorer.stations
get all the stations currently selected
Explorer.stats
get stats for the currently selected values
Explorer.tranges
get all the time range values currently selected
Explorer.varcodes
get all the varcode values currently selected
Explorer.query_summary()
Get the currently selected Explorer summary information; returns a Cursor
Explorer.query_summary_all()
Get all the Explorer summary information; returns a Cursor
Explorer.rebuild()

Empty the Explorer and start adding new data to it.

Returns an ExplorerUpdate context manager object that can be used to add data to the explorer in a single transaction.

Explorer.set_filter()
Set a new filter, updating all browsing data
Explorer.to_json()

Serialize the contents of this explorer to JSON.

Only the global summary is serialized: the current query is not preserved.

Explorer.update()

Start adding new data to the Explorer without clearing it first.

Returns an ExplorerUpdate context manager object that can be used to add data to the explorer in a single transaction.

dballe.DBExplorer

Browser for a summary of DB-All-e database of message contents

Members

DBExplorer.all_levels
get all level values
DBExplorer.all_reports
get all report values
DBExplorer.all_stations
get all stations
DBExplorer.all_stats
get the stats for all values
DBExplorer.all_tranges
get all time range values
DBExplorer.all_varcodes
get all varcode values
DBExplorer.levels
get all the level values currently selected
DBExplorer.reports
get all the report values currently selected
DBExplorer.stations
get all the stations currently selected
DBExplorer.stats
get stats for the currently selected values
DBExplorer.tranges
get all the time range values currently selected
DBExplorer.varcodes
get all the varcode values currently selected
DBExplorer.query_summary()
Get the currently selected Explorer summary information; returns a Cursor
DBExplorer.query_summary_all()
Get all the Explorer summary information; returns a Cursor
DBExplorer.rebuild()

Empty the Explorer and start adding new data to it.

Returns an ExplorerUpdate context manager object that can be used to add data to the explorer in a single transaction.

DBExplorer.set_filter()
Set a new filter, updating all browsing data
DBExplorer.to_json()

Serialize the contents of this explorer to JSON.

Only the global summary is serialized: the current query is not preserved.

DBExplorer.update()

Start adding new data to the Explorer without clearing it first.

Returns an ExplorerUpdate context manager object that can be used to add data to the explorer in a single transaction.

dballe.ExplorerUpdate

Manage updates to an Explorer

Members

ExplorerUpdate.add_db()
Add the summary of the contents of the given database to the Explorer.
ExplorerUpdate.add_explorer()
Add the contents of the given Explorer or DBExplorer to the Explorer.
ExplorerUpdate.add_json()
Add the contents of the given JSON string to the Explorer.
ExplorerUpdate.add_messages()

Add dballe.Message objects to the explorer.

It takes the same messages argument of dballe.DB.import_messages

dballe.DBExplorerUpdate

Manage updates to an Explorer

Members

DBExplorerUpdate.add_db()
Add the summary of the contents of the given database to the Explorer.
DBExplorerUpdate.add_explorer()
Add the contents of the given Explorer or DBExplorer to the Explorer.
DBExplorerUpdate.add_json()
Add the contents of the given JSON string to the Explorer.
DBExplorerUpdate.add_messages()

Add dballe.Message objects to the explorer.

It takes the same messages argument of dballe.DB.import_messages

The volnd API

volnd is a way of extracting entire matrixes of data out of a DB-All.e database.

This module allows to extract multidimensional matrixes of data given a list of dimension definitions. Every dimension definition defines what kind of data goes along that dimension.

Dimension definitions can be shared across different extracted matrixes and multiple extractions, allowing to have different matrixes whose indexes have the same meaning.

This example code extracts temperatures in a station by datetime matrix:

query = dict()
query["var"] = "B12001"
query["rep_memo"] = "synop"
query["level"] = (105, 2)
query["trange"] = (0,)
vars = read(self.db.query(query), (AnaIndex(), DateTimeIndex()))
data = vars["B12001"]
# Data is now a 2-dimensional Masked Array with the data
#
# Information about what values correspond to an index in the various
# directions can be accessed in data.dims, which contains one list per
# dimension with all the information corresponding to every index.
print("Ana dimension is", len(data.dims[0]), "items long")
print("Datetime dimension is", len(data.dims[1]), "items long")
print("First 10 stations along the Ana dimension:", data.dims[0][:10])
print("First 10 datetimes along the DateTime dimension:", data.dims[1][:10])

This is the list of dimensions supported by dballe.volnd:

AnaIndex

Index for stations, as they come out of the database.

The constructor syntax is: AnaIndex(shared=True, frozen=False, start=None).

The index saves all stations as AnaIndexEntry tuples, in the same order as they come out of the database.

NetworkIndex

Index for networks, as they come out of the database.

The constructor syntax is: NetworkIndex(shared=True, frozen=False, start=None).

The index saves all networks as NetworkIndexEntry tuples, in the same order as they come out of the database.

LevelIndex

Index for levels, as they come out of the database

The constructor syntax is: LevelIndex(shared=True, frozen=False, start=None).

The index saves all levels as dballe.Level tuples, in the same order as they come out of the database.

TimeRangeIndex

Index for time ranges, as they come out of the database.

The constructor syntax is: TimeRangeIndex(shared=True, frozen=False, start=None).

The index saves all time ranges as dballe.TimeRange tuples, in the same order as they come out of the database.

DateTimeIndex

Index for datetimes, as they come out of the database.

The constructor syntax is: DateTimeIndex(shared=True, frozen=False, start=None).

The index saves all datetime values as datetime.datetime objects, in the same order as they come out of the database.

IntervalIndex

Index into equally spaced points in time, starting at start, with a point every step time.

Index points are at fixed time intervals, and data is acquired in one point only if it is within a given tolerance from the interval.

The constructor syntax is: IntervalIndex(start, step, tolerance=0, end=None, shared=True, frozen=False).

start is a datetime.datetime object giving the starting time of the time interval of this index.

step is a datetime.timedelta object with the interval between sampling points.

tolerance is a datetime.timedelta object specifying the maximum allowed interval between a datum datetime and the sampling step. If the interval is bigger than the tolerance, the data is discarded.

end is an optional datetime.datetime object giving the ending time of the time interval of the index. If omitted, the index will end at the latest accepted datum coming out of the database.

The data object used by AnaIndex is:

AnaIndexEntry

AnaIndex entry, with various data about a single station.

It is a named tuple of 4 values:
  • id: station id
  • lat: latitude
  • lon: longitude
  • ident: mobile station identifier, or None

The extraction is done using the dballe.volnd.read function:

read(cursor, dims, filter=None, checkConflicts=True, attributes=None)

cursor is a dballe.Cursor resulting from a dballe query

dims is the sequence of indexes to use for shaping the data matrixes

filter is an optional filter function that can be used to discard values from the query: if filter is not None, it will be called for every output record and if it returns False, the record will be discarded

checkConflicts tells if we should raise an exception if two values from the database would fill in the same position in the matrix

attributes tells if we should read attributes as well: if it is None, no attributes will be read; if it is True, all attributes will be read; if it is a sequence, then it is the sequence of attributes that should be read.

The result of dballe.volnd.read is a dict mapping output variable names to a dballe.volnd.Data object with the results. All the Data objects share their indexes unless the xxx-Index definitions have been created with shared=False.

This is the dballe.volnd.Data class documentation:

Data

Container for collecting variable data. It contains the variable data array and the dimension indexes.

If v is a Data object, you can access the tuple with the dimensions as v.dims, and the masked array with the values as v.vals.

The methods of dballe.volnd.Data are:

Data.append(…)

Collect a new value from the given dballe record.

You need to call finalise() before the values can be used.

Data.appendAttrs(…)

Collect attributes to append to the record.

You need to call finalise() before the values can be used.

Data.finalise(…)
Stop collecting values and create a masked array with all the values collected so far.