The DB-All.e Python bindings provide 2 levels of access to a DB-All.e database: a complete API similar to the Fortran and C++ API, and a high-level API called volnd that allows to automatically export matrices of data out of the database.
Contents
The 'dballe' module has a few global methods:
- dballe.describe_level(ltype1: int, l1: int=None, ltype2: int=None, l2: int=None) -> str
- Return a string description for a level
- dballe.describe_trange(pind: int, p1: int=None, p2: int=None) -> str
- Return a string description for a time range
- dballe.var(code, val: Any=None) -> dballe.Var
- Query the DB-All.e variable table returning a Var, optionally initialized with a value
- dballe.varinfo(str) -> str
- Query the DB-All.e variable table returning a Varinfo
and several classes, documented in their own sections.
Var holds a measured value, which can be integer, float or string, and a dballe.Varinfo with all available information (description, unit, precision, ...) related to it.
Var objects can be created from a dballe.Varinfo object, and an optional value. Omitting the value creates an unset variable.
Examples:
v = dballe.Var(table["B12101"], 32.5) # v.info returns detailed informations about the variable in a Varinfo object. print("%s: %s %s %s" % (v.code, str(v), v.info.unit, v.info.desc))
- Var.code
- variable code
- Var.info
- Varinfo for this variable
- Var.isset
- true if the value is set
- Var.enq() -> Union[str, float, int]
- get the value of the variable, as int, float or str according the variable definition
- Var.enqa(code: str) -> Optional[dballe.Var]
- get the variable for the attribute with the given code, or None if not found
- Var.enqc() -> str
- get the value of the variable, as a str
- Var.enqd() -> float
- get the value of the variable, as a float
- Var.enqi() -> int
- get the value of the variable, as an int
- Var.format(default: str=) -> str
- return a string with the formatted value of the variable
- Var.get(default: Any=None) -> Union[str, float, long, Any]
- get the value of the variable, with a default if it is unset
- Var.get_attrs() -> List[dballe.Var]
- get the attributes of this variable
- Var.seta(var: dballe.Var)
- set an attribute in the variable
- Var.unseta(code: str)
- unset the given attribute from the variable
Varinfo object holds all possible information about a variable, such as its measurement unit, description and number of significant digits.
Varinfo objects cannot be instantiated directly, and are created by querying dballe.Vartable objects.
- Varinfo.bit_len
- number of bits used to encode the value in BUFR
- Varinfo.bit_ref
- reference value added after scaling, for BUFR decoding
- Varinfo.code
- variable code
- Varinfo.desc
- description
- Varinfo.len
- number of significant digits
- Varinfo.scale
- scale of the value as a power of 10
- Varinfo.type
- return a string describing the type of the variable (string, binary, integer, decimal)
- Varinfo.unit
- measurement unit
Collection of Varinfo objects indexed by WMO BUFR/CREX table B code.
A Vartable is instantiated by the name (without extension) of the table file installed in wreport's data directory (normally, /usr/share/wreport/):
table = dballe.Vartable("B0000000000000023000") print(table["B12101"].desc) for i in table: print(i.code, i.desc)
- Vartable.pathname
- name of the table
- Vartable.get_bufr(basename: str=None, originating_centre: int=0, originating_subcentre: int=0,master_table_number: int=0, master_table_version_number: int=None, master_table_version_number_local: int=0) -> dballe.Vartable
Look up a table B file using the information given, then load BUFR information from it.
You need to provide either basename or master_table_version_number.
- Vartable.get_crex(basename: str=None, edition_number=2, originating_centre: int=0, originating_subcentre: int=0,master_table_number: int=0, master_table_version_number: int=None,master_table_version_number_bufr: int=None, master_table_version_number_local: int=0) -> dballe.Vartable
Look up a table B file using the information given, then load CREX information from it.
You need to provide either basename or master_table_version_number or master_table_version_number_bufr.
- Vartable.load_bufr(pathname: str) -> dballe.Vartable
- Load BUFR information from a Table B file and return it as a dballe.Vartable.
- Vartable.load_crex(pathname: str) -> dballe.Vartable
- Load CREX information from a Table B file and return it as a dballe.Vartable.
Level or layer.
Constructor: Level(ltype1: int=None, l1: int=None, ltype2: int=None, l2: int=None)
- Level.l1
- value of the level or of the first layer
- Level.l2
- value of the second layer
- Level.ltype1
- type of the level or of the first layer
- Level.ltype2
- type of the second layer
Time range.
Constructor: Trange(pind: int=None, p1: int=None, p2: int=None)
- Trange.p1
- Time range P1 indicator
- Trange.p2
- Time range P2 indicator
- Trange.pind
- Time range type indicator
Station information.
Constructor: Station(report: str, lat: float, lon: float, ident: str=None)
- Station.ident
- mobile station identifier
- Station.lat
- station latitude
- Station.lon
- station longitude
- Station.report
- report for this station
Station information with database ID.
Constructor: Station(report: str, id: int, lat: float, lon: float, ident: str=None)
- DBStation.id
- database ID for this station
- DBStation.ident
- mobile station identifier
- DBStation.lat
- station latitude
- DBStation.lon
- station longitude
- DBStation.report
- report for this station
key-value representation of a value with its associated metadata
Binary message.
This is basically a simple wrapper around a bytes() object, providing extra information about the filename, offset and index where the message data was read. Is it used by dballe.File to return the binary messages it reads.
- BinaryMessage.encoding
- message encoding
- BinaryMessage.index
- index of the message in the input file, or None if unknown
- BinaryMessage.offset
- offset of the message in the input file, or None if unknown
- BinaryMessage.pathname
- pathname of the file the message came from, or None if unknown
Read-only access to files with weather bulletins in BUFR or CREX format.
No write functions are supported: to write files, you can simply write dballe.BinaryMessage objects or encoded messages to normal Python files.
Constructor: File(file: Union[str, File], encoding: str=None)
file can be a file name, or a file-like object. If a file-like object supports fileno(), that file descriptor is dup()-ed and used for efficient reading. Otherwise, file.read() is called to load the data to read in memory.
If encoding is omitted, it is auto detected by looking at the first byte of the file only. Files with leading padding data will not be detected properly, and you need to explicitly specify the encoding to read them.
Example usage:
with dballe.File("test.bufr", "BUFR") as f: for binmsg in f: print("#{m.index}: {m.pathname}:{m.offset}: {m.encoding} message".format(m=binmsg))
- File.encoding
- get the file encoding
- File.name
- get the file name
The contents of a decoded BUFR or CREX message.
DB-All.e can interpret the contents of most weather messages commonly in use, and represent them as variables identified by dballe.Level, dballe.Trange, datetime, coordinates, network, and mobile station identifier.
A message contains only one reference station (coordinates, network, mobile station identifier), only one reference datetime, and many (level, trange, varcode, value) variables.
Variables that describe the station are accessible using None for level and trange.
Constructor: Message(type: str)
type is a string identifying the message type, and it will affect how the message will be encoded by the exporter.
Example usage:
importer = dballe.Importer("BUFR") with importer.from_file("test.bufr") as f: for msg in f: print("{m.report},{m.coords},{m.ident},{m.datetime},{m.type}".format(m=msg))
- Message.coords
- message coordinates
- Message.datetime
- message datetime
- Message.ident
- message mobile station identifier
- Message.report
- message report
- Message.type
- message type
- Message.get(level: dballe.Level, trange: dballe.Trange, code: str) -> Union[dballe.Var, None]
- Get a Var given its level, timerange, and varcode; returns None if not found
- Message.get_named(name: str) -> Union[dballe.Var, None]
- Get a Var given its shortcut name; returns None if not found
- Message.query_data(query: Dict[str, Any]) -> dballe.CursorData
- Query the variables in the message
- Message.query_station_data(query: Dict[str, Any]) -> dballe.CursorStationData
- Query the station variables in the message
- Message.query_stations(query: Dict[str, Any]) -> dballe.CursorStation
- Query the station data in the message
- Message.set(level: dballe.Level, trange: dballe.Trange, var: dballe.Var)
- Set a Var given level and timerange
- Message.set_named(name: str, var: dballe.Var)
- Set a Var given its shortcut name
Message importer.
This is the engine that decodes binary messages and interprets their contents using a uniform data model.
Note that one binary message is often decoded to multiple data messages, in case, for example, of compressed BUFR files.
Constructor: Importer(encoding: str, simplified: bool=True)
encoding can be "BUFR"
or "CREX"
.
simplified control whether messages are constructed using standard levels and time ranges, or using the exact levels and time ranges contained in the input. For example, a simplified intepretation of a synop message will place the temperature at 2M above ground, regardless of the reported sensor height. A non-simplified import will place the temperature reading at the reported sensor height.
When a message is imported in simplified mode, the actual context information will be stored as data attributes.
Example usage:
importer = dballe.Importer("BUFR") with importer.from_file("test.bufr") as f: for msg in f: print("{m.report},{m.coords},{m.ident},{m.datetime},{m.type}".format(m=msg)) importer = dballe.Importer("BUFR") with dbale.File("test.bufr") as f: for binmsg in f: msgs = importer.from_binary(binmsg) for msg in msgs: print("#{b.index}: {m.report},{m.coords},{m.ident},{m.datetime},{m.type}".format(b=binmsg, m=msg))
- Importer.from_binary(binmsg: dballe.BinaryMessage) -> Sequence[dballe.BinaryMessage]
- Decode a BinaryMessage to a tuple of dballe.Message objects
- Importer.from_file(file: Union[dballe.File, str, File]) -> dballe.ImporterFile
Wrap a dballe.File into a sequence of tuples of dballe.Message objects.
file can be a dballe.File, a file name, or a file-like object. A dballe.File is automatically constructed if needed, using the importer encoding.
Message importer iterating over the contents of a a dballe.File.
This is never instantiated explicitly, but is returned by Importer.from_file().
It can be used in a context manager, and it is an iterable that yields tuples of dballe.Message objects.
Message exporter.
This is the engine that can reconstruct a standard BUFR or CREX message from the contents of a dballe.Message.
- Exporter.to_binary(contents: Union[dballe.Message, Sequence[dballe.Message], Iterable[dballe.Message]]) -> bytes
- Encode a dballe.Message or a sequence of dballe.Message into a bytes object.
DB-All.e database access.
Many methods are the same in dballe.DB and dballe.Transaction. The versions in dballe.DB are implemented by automatically creating a temporary transaction and running the equivalent dballe.Transaction method inside it.
dballe.DB objects are not constructed explicitly, but via one of the DB.connect or DB.connect_test class methods.
Examples:
# Connect to a database and run a query db = dballe.DB.connect_from_file("db.sqlite") query = {latmin=44.0, latmax=45.0, lonmin=11.0, lonmax=12.0} # The result is a dballe.Cursor (dballe.CursorData in this case), which can # be iterated to get results as dict objects. for row in db.query_data(query): print(row["lat"], row["lon"], row["var"], row.var().format("undefined")) # Insert 2 new variables in the database db.insert_data({ "lat": 44.5, "lon": 11.4, "level": dballe.Level(1), "trange": dballe.Trange(254), "date": datetime.datetime(2013, 4, 25, 12, 0, 0), "B11101": 22.4, "B12103": 17.2, })
- DB.attr_insert_data(varid: int, attrs: Dict[str, Any])
- Insert new attributes into the database
- DB.attr_insert_station(varid: int, attrs: Dict[str, Any])
- Insert new attributes into the database
- DB.attr_query_data(varid: int) -> Dict[str, Any]
- query data attributes
- DB.attr_query_station(varid: int) -> Dict[str, Any]
- query station data attributes
- DB.attr_remove_data(varid: int, attrs: Iterable[str])
- Remove attributes from data variables
- DB.attr_remove_station(varid: int, attrs: Iterable[str])
- Remove attributes from station variables
- DB.connect(url: str) -> dballe.DB
- create a DB to access a database identified by a DB-All.e URL
- DB.connect_from_file(name: str) -> dballe.DB
- create a DB to access a SQLite file
- DB.connect_from_url(url: str) -> dballe.DB
- create a DB to access a database identified by a DB-All.e URL (deprecated, use connect instead)
- DB.connect_test() -> dballe.DB
- Create a DB for running the test suite, as configured in the test environment
- DB.disappear()
- Remove all DB-All.e tables and data from the database, if possible
- DB.get_default_format() -> str
- get the default DB format
- DB.import_messages(messages: Union[dballe.Message, Sequence[dballe.Message], Iterable[dballe.Message], dballe.ImporterFile], report: str=None, import_attributres: bool=False, update_station: bool=False, overwrite: bool=False)
Import one or more Messages into the database.
- messages can be:
- a dballe.Message object
- a sequence or iterable of dballe.Message objects
- a dballe.ImporterFile that generates a sequence of dballe.Message objects
report is the network name to use for importing the data. If left to None, the network is selected automatically from the message type
import_attributes if set to True, requests the variable attributes to also be imported.
update_station, if set to True, station information is merged with existing one in the database. If false (default), station information is imported only when the station did not exist in the database.
overwrite, if set to True, causes existing information already in the database to be overwritten. If false (default), trying to import a message which contains data already present in the database causes the import to fail.
varlist, if set to a string in the same format as the varlist query parameter, only imports data whose varcode is in the list.
- DB.insert_data(record: Union[Dict[str, Any], dballe.Cursor], can_replace: bool=False, can_add_stations: bool=False) -> Dict[str, int]
Insert data values in the database
The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.
- DB.insert_station_data(record: Union[Dict[str, Any], dballe.Cursor], can_replace: bool=False, can_add_stations: bool=False) -> Dict[str, int]
Insert station values in the database
The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.
- DB.is_url(url: str) -> bool
- Checks if a string looks like a DB-All.e DB url
- DB.query_data(query: Dict[str, Any]) -> dballe.CursorData
- Query the variables in the database
- DB.query_messages(query: Dict[str, Any]) -> dballe.CursorMessage
- Query the database returning the matching data as Message objects
- DB.query_station_data(query: Dict[str, Any]) -> dballe.CursorStationData
- Query the station variables in the database
- DB.query_stations(query: Dict[str, Any]) -> dballe.CursorStation
- Query the station archive in the database
- DB.query_summary(query: Dict[str, Any]) -> dballe.CursorSummary
- Query the summary of the results of a query
- DB.remove(query: Dict[str, Any])
- Remove data variables from the database (deprecated)
- DB.remove_all()
- Remove all data from the database
- DB.remove_data(query: Dict[str, Any])
- Remove data variables from the database
- DB.remove_station_data(query: Dict[str, Any])
- Remove station variables from the database
- DB.reset(repinfo_file: str=None)
- Reset the database, removing all existing Db-All.e tables and re-creating them empty.
- DB.set_default_format(format: str)
- set the default DB format
- DB.transaction(readonly: bool=False) -> dballe.Transaction
- Create a new database transaction
- DB.vacuum()
- Perform database cleanup operations
DB-All.e transaction
A Transaction is used to execute DB operations in an all-or-nothing fashion. In fact, most DB methods are implemented using a short-lived temporary transaction.
You cannot have more than one active dballe.Transaction for each dballe.DB. An attempt to start a second one will result in an exception being raised. Note that dballe.DB functions like insert_data or export_to_file create a temporary transaction to run, and so they will also fail if a transaction is currently open. The general idea is that all database work should be done inside a transaction.
Transactions run using the REPEATABLE READ isolation level of the underlying database. This usually means that modifications performed inside a transaction are not visible to other database connections until the transaction is committed. If a transaction is rolled back, all changes done with it are undone.
Transactions can also be used as context managers, like this:
with db.transaction() as t: for i in range(10): t.insert({ "lat": 44.5 + i, "lon": 11.4 + i, "level": (1,), "trange": (254,), "date": datetime.datetime(2013, 4, 25, 12, 0, 0), "B11101": 22.4 + i, "B12103": 17.2 })
The dballe.Transaction methods are the same as those in dballe.DB. The version in dballe.DB is implemented by automatically creating a temporary transaction and running the dballe.Transaction method inside it.
- Transaction.attr_insert_data(varid: int, attrs: Dict[str, Any])
- Insert new attributes into the database
- Transaction.attr_insert_station(varid: int, attrs: Dict[str, Any])
- Insert new attributes into the database
- Transaction.attr_query_data(varid: int) -> Dict[str, Any]
- query data attributes
- Transaction.attr_query_station(varid: int) -> Dict[str, Any]
- query station data attributes
- Transaction.attr_remove_data(varid: int, attrs: Iterable[str])
- Remove attributes from data variables
- Transaction.attr_remove_station(varid: int, attrs: Iterable[str])
- Remove attributes from station variables
- Transaction.commit()
- commit the transaction
- Transaction.import_messages(messages: Union[dballe.Message, Sequence[dballe.Message], Iterable[dballe.Message], dballe.ImporterFile], report: str=None, import_attributres: bool=False, update_station: bool=False, overwrite: bool=False)
Import one or more Messages into the database.
- messages can be:
- a dballe.Message object
- a sequence or iterable of dballe.Message objects
- a dballe.ImporterFile that generates a sequence of dballe.Message objects
report is the network name to use for importing the data. If left to None, the network is selected automatically from the message type
import_attributes if set to True, requests the variable attributes to also be imported.
update_station, if set to True, station information is merged with existing one in the database. If false (default), station information is imported only when the station did not exist in the database.
overwrite, if set to True, causes existing information already in the database to be overwritten. If false (default), trying to import a message which contains data already present in the database causes the import to fail.
varlist, if set to a string in the same format as the varlist query parameter, only imports data whose varcode is in the list.
- Transaction.insert_data(record: Union[Dict[str, Any], dballe.Cursor], can_replace: bool=False, can_add_stations: bool=False) -> Dict[str, int]
Insert data values in the database
The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.
- Transaction.insert_station_data(record: Union[Dict[str, Any], dballe.Cursor], can_replace: bool=False, can_add_stations: bool=False) -> Dict[str, int]
Insert station values in the database
The return value is a dict that always contains ana_id mapped to the station ID just inserted, and an entry for each varcode inserted mapping to the database ID of its value.
- Transaction.query_data(query: Dict[str, Any]) -> dballe.CursorData
- Query the variables in the database
- Transaction.query_messages(query: Dict[str, Any]) -> dballe.CursorMessage
- Query the database returning the matching data as Message objects
- Transaction.query_station_data(query: Dict[str, Any]) -> dballe.CursorStationData
- Query the station variables in the database
- Transaction.query_stations(query: Dict[str, Any]) -> dballe.CursorStation
- Query the station archive in the database
- Transaction.query_summary(query: Dict[str, Any]) -> dballe.CursorSummary
- Query the summary of the results of a query
- Transaction.remove(query: Dict[str, Any])
- Remove data variables from the database (deprecated)
- Transaction.remove_all()
- Remove all data from the database
- Transaction.remove_data(query: Dict[str, Any])
- Remove data variables from the database
- Transaction.remove_station_data(query: Dict[str, Any])
- Remove station variables from the database
- Transaction.rollback()
- roll back the transaction
A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.
A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.
For example:
with self.db.query_data({...}) as cur: for row in cur: print("Station:", cur["station"]) print("Var:", cur["var"], cur.enqf(cur["var"]))
- CursorStation.query
- return a dict with a query to select exactly the current value at this cursor
- CursorStation.remaining
- number of results still to be returned
- CursorStation.enqd(key: str) -> Union[float, None]
- Return the float value for a keyword
- CursorStation.enqf(key: str) -> Union[str, None]
- Return the formatted string value for a keyword
- CursorStation.enqi(key: str) -> Union[int, None]
- Return the integer value for a keyword
- CursorStation.enqs(key: str) -> Union[str, None]
- Return the string value for a keyword
A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.
A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.
For example:
with self.db.query_data({...}) as cur: for row in cur: print("Station:", cur["station"]) print("Var:", cur["var"], cur.enqf(cur["var"]))
- CursorStationData.data
- return a dballe.Data which can be used to insert into a database the current cursor value
- CursorStationData.data_dict
- return a dict which can be used to insert into a database the current cursor value
- CursorStationData.remaining
- number of results still to be returned
- CursorStationData.enqd(key: str) -> Union[float, None]
- Return the float value for a keyword
- CursorStationData.enqf(key: str) -> Union[str, None]
- Return the formatted string value for a keyword
- CursorStationData.enqi(key: str) -> Union[int, None]
- Return the integer value for a keyword
- CursorStationData.enqs(key: str) -> Union[str, None]
- Return the string value for a keyword
A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.
A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.
For example:
with self.db.query_data({...}) as cur: for row in cur: print("Station:", cur["station"]) print("Var:", cur["var"], cur.enqf(cur["var"]))
- CursorData.data
- return a dballe.Data which can be used to insert into a database the current cursor value
- CursorData.data_dict
- return a dict which can be used to insert into a database the current cursor value
- CursorData.remaining
- number of results still to be returned
- CursorData.enqd(key: str) -> Union[float, None]
- Return the float value for a keyword
- CursorData.enqf(key: str) -> Union[str, None]
- Return the formatted string value for a keyword
- CursorData.enqi(key: str) -> Union[int, None]
- Return the integer value for a keyword
- CursorData.enqs(key: str) -> Union[str, None]
- Return the string value for a keyword
A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.
A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.
For example:
with self.db.query_data({...}) as cur: for row in cur: print("Station:", cur["station"]) print("Var:", cur["var"], cur.enqf(cur["var"]))
- CursorStationDB.query
- return a dict with a query to select exactly the current value at this cursor
- CursorStationDB.remaining
- number of results still to be returned
- CursorStationDB.enqd(key: str) -> Union[float, None]
- Return the float value for a keyword
- CursorStationDB.enqf(key: str) -> Union[str, None]
- Return the formatted string value for a keyword
- CursorStationDB.enqi(key: str) -> Union[int, None]
- Return the integer value for a keyword
- CursorStationDB.enqs(key: str) -> Union[str, None]
- Return the string value for a keyword
- CursorStationDB.remove()
- Remove the data currently addressed by the cursor
A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.
A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.
For example:
with self.db.query_data({...}) as cur: for row in cur: print("Station:", cur["station"]) print("Var:", cur["var"], cur.enqf(cur["var"]))
- CursorStationDataDB.data
- return a dballe.Data which can be used to insert into a database the current cursor value
- CursorStationDataDB.data_dict
- return a dict which can be used to insert into a database the current cursor value
- CursorStationDataDB.remaining
- number of results still to be returned
- CursorStationDataDB.enqd(key: str) -> Union[float, None]
- Return the float value for a keyword
- CursorStationDataDB.enqf(key: str) -> Union[str, None]
- Return the formatted string value for a keyword
- CursorStationDataDB.enqi(key: str) -> Union[int, None]
- Return the integer value for a keyword
- CursorStationDataDB.enqs(key: str) -> Union[str, None]
- Return the string value for a keyword
- CursorStationDataDB.insert_attrs(attrs: Dict[str, Any])
- Insert or update attributes for the current variable
- CursorStationDataDB.remove()
- Remove the data currently addressed by the cursor
- CursorStationDataDB.remove_attrs(attrs: Iterable[str])
- Remove attributes from the current variable
A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.
A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.
For example:
with self.db.query_data({...}) as cur: for row in cur: print("Station:", cur["station"]) print("Var:", cur["var"], cur.enqf(cur["var"]))
- CursorDataDB.data
- return a dballe.Data which can be used to insert into a database the current cursor value
- CursorDataDB.data_dict
- return a dict which can be used to insert into a database the current cursor value
- CursorDataDB.remaining
- number of results still to be returned
- CursorDataDB.enqd(key: str) -> Union[float, None]
- Return the float value for a keyword
- CursorDataDB.enqf(key: str) -> Union[str, None]
- Return the formatted string value for a keyword
- CursorDataDB.enqi(key: str) -> Union[int, None]
- Return the integer value for a keyword
- CursorDataDB.enqs(key: str) -> Union[str, None]
- Return the string value for a keyword
- CursorDataDB.insert_attrs(attrs: Dict[str, Any])
- Insert or update attributes for the current variable
- CursorDataDB.remove()
- Remove the data currently addressed by the cursor
- CursorDataDB.remove_attrs(attrs: Iterable[str])
- Remove attributes from the current variable
A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.
A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.
For example:
with self.db.query_data({...}) as cur: for row in cur: print("Station:", cur["station"]) print("Var:", cur["var"], cur.enqf(cur["var"]))
- CursorSummaryDB.query
- return a dict with a query to select exactly the current value at this cursor
- CursorSummaryDB.remaining
- number of results still to be returned
- CursorSummaryDB.enqd(key: str) -> Union[float, None]
- Return the float value for a keyword
- CursorSummaryDB.enqf(key: str) -> Union[str, None]
- Return the formatted string value for a keyword
- CursorSummaryDB.enqi(key: str) -> Union[int, None]
- Return the integer value for a keyword
- CursorSummaryDB.enqs(key: str) -> Union[str, None]
- Return the string value for a keyword
- CursorSummaryDB.remove()
- Remove the data currently addressed by the cursor
A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.
A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.
For example:
with self.db.query_data({...}) as cur: for row in cur: print("Station:", cur["station"]) print("Var:", cur["var"], cur.enqf(cur["var"]))
- CursorSummarySummary.query
- return a dict with a query to select exactly the current value at this cursor
- CursorSummarySummary.remaining
- number of results still to be returned
- CursorSummarySummary.enqd(key: str) -> Union[float, None]
- Return the float value for a keyword
- CursorSummarySummary.enqf(key: str) -> Union[str, None]
- Return the formatted string value for a keyword
- CursorSummarySummary.enqi(key: str) -> Union[int, None]
- Return the integer value for a keyword
- CursorSummarySummary.enqs(key: str) -> Union[str, None]
- Return the string value for a keyword
A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.
A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.
For example:
with self.db.query_data({...}) as cur: for row in cur: print("Station:", cur["station"]) print("Var:", cur["var"], cur.enqf(cur["var"]))
- CursorSummaryDBSummary.query
- return a dict with a query to select exactly the current value at this cursor
- CursorSummaryDBSummary.remaining
- number of results still to be returned
- CursorSummaryDBSummary.enqd(key: str) -> Union[float, None]
- Return the float value for a keyword
- CursorSummaryDBSummary.enqf(key: str) -> Union[str, None]
- Return the formatted string value for a keyword
- CursorSummaryDBSummary.enqi(key: str) -> Union[int, None]
- Return the integer value for a keyword
- CursorSummaryDBSummary.enqs(key: str) -> Union[str, None]
- Return the string value for a keyword
A Cursor is the result of database queries. It is generally iterated through the contents of the result. Each iteration returns the cursor itself, that can be used to access the result values.
A Cursor works like a read-only dict to access data from the current result. The list of available keys is documented in doc/fapi_parms.md.
For example:
with self.db.query_data({...}) as cur: for row in cur: print("Station:", cur["station"]) print("Var:", cur["var"], cur.enqf(cur["var"]))
- CursorMessage.message
- dballe.Message object with the current message
- CursorMessage.query
- return a dict with a query to select exactly the current value at this cursor
- CursorMessage.remaining
- number of results still to be returned
- CursorMessage.enqd(key: str) -> Union[float, None]
- Return the float value for a keyword
- CursorMessage.enqf(key: str) -> Union[str, None]
- Return the formatted string value for a keyword
- CursorMessage.enqi(key: str) -> Union[int, None]
- Return the integer value for a keyword
- CursorMessage.enqs(key: str) -> Union[str, None]
- Return the string value for a keyword
Browser for a summary of DB-All-e database of message contents
- Explorer.all_levels
- get all level values
- Explorer.all_reports
- get all report values
- Explorer.all_stations
- get all stations
- Explorer.all_stats
- get the stats for all values
- Explorer.all_tranges
- get all time range values
- Explorer.all_varcodes
- get all varcode values
- Explorer.levels
- get all the level values currently selected
- Explorer.reports
- get all the report values currently selected
- Explorer.stations
- get all the stations currently selected
- Explorer.stats
- get stats for the currently selected values
- Explorer.tranges
- get all the time range values currently selected
- Explorer.varcodes
- get all the varcode values currently selected
- Explorer.query_summary()
- Get the currently selected Explorer summary information; returns a Cursor
- Explorer.query_summary_all()
- Get all the Explorer summary information; returns a Cursor
- Explorer.rebuild()
Empty the Explorer and start adding new data to it.
Returns an ExplorerUpdate context manager object that can be used to add data to the explorer in a single transaction.
- Explorer.set_filter()
- Set a new filter, updating all browsing data
- Explorer.to_json()
Serialize the contents of this explorer to JSON.
Only the global summary is serialized: the current query is not preserved.
- Explorer.update()
Start adding new data to the Explorer without clearing it first.
Returns an ExplorerUpdate context manager object that can be used to add data to the explorer in a single transaction.
Browser for a summary of DB-All-e database of message contents
- DBExplorer.all_levels
- get all level values
- DBExplorer.all_reports
- get all report values
- DBExplorer.all_stations
- get all stations
- DBExplorer.all_stats
- get the stats for all values
- DBExplorer.all_tranges
- get all time range values
- DBExplorer.all_varcodes
- get all varcode values
- DBExplorer.levels
- get all the level values currently selected
- DBExplorer.reports
- get all the report values currently selected
- DBExplorer.stations
- get all the stations currently selected
- DBExplorer.stats
- get stats for the currently selected values
- DBExplorer.tranges
- get all the time range values currently selected
- DBExplorer.varcodes
- get all the varcode values currently selected
- DBExplorer.query_summary()
- Get the currently selected Explorer summary information; returns a Cursor
- DBExplorer.query_summary_all()
- Get all the Explorer summary information; returns a Cursor
- DBExplorer.rebuild()
Empty the Explorer and start adding new data to it.
Returns an ExplorerUpdate context manager object that can be used to add data to the explorer in a single transaction.
- DBExplorer.set_filter()
- Set a new filter, updating all browsing data
- DBExplorer.to_json()
Serialize the contents of this explorer to JSON.
Only the global summary is serialized: the current query is not preserved.
- DBExplorer.update()
Start adding new data to the Explorer without clearing it first.
Returns an ExplorerUpdate context manager object that can be used to add data to the explorer in a single transaction.
Manage updates to an Explorer
- ExplorerUpdate.add_db()
- Add the summary of the contents of the given database to the Explorer.
- ExplorerUpdate.add_explorer()
- Add the contents of the given Explorer or DBExplorer to the Explorer.
- ExplorerUpdate.add_json()
- Add the contents of the given JSON string to the Explorer.
- ExplorerUpdate.add_messages()
Add dballe.Message objects to the explorer.
It takes the same messages argument of dballe.DB.import_messages
Manage updates to an Explorer
- DBExplorerUpdate.add_db()
- Add the summary of the contents of the given database to the Explorer.
- DBExplorerUpdate.add_explorer()
- Add the contents of the given Explorer or DBExplorer to the Explorer.
- DBExplorerUpdate.add_json()
- Add the contents of the given JSON string to the Explorer.
- DBExplorerUpdate.add_messages()
Add dballe.Message objects to the explorer.
It takes the same messages argument of dballe.DB.import_messages
volnd is a way of extracting entire matrixes of data out of a DB-All.e database.
This module allows to extract multidimensional matrixes of data given a list of dimension definitions. Every dimension definition defines what kind of data goes along that dimension.
Dimension definitions can be shared across different extracted matrixes and multiple extractions, allowing to have different matrixes whose indexes have the same meaning.
This example code extracts temperatures in a station by datetime matrix:
query = dict() query["var"] = "B12001" query["rep_memo"] = "synop" query["level"] = (105, 2) query["trange"] = (0,) vars = read(self.db.query(query), (AnaIndex(), DateTimeIndex())) data = vars["B12001"] # Data is now a 2-dimensional Masked Array with the data # # Information about what values correspond to an index in the various # directions can be accessed in data.dims, which contains one list per # dimension with all the information corresponding to every index. print("Ana dimension is", len(data.dims[0]), "items long") print("Datetime dimension is", len(data.dims[1]), "items long") print("First 10 stations along the Ana dimension:", data.dims[0][:10]) print("First 10 datetimes along the DateTime dimension:", data.dims[1][:10])
This is the list of dimensions supported by dballe.volnd:
- AnaIndex
Index for stations, as they come out of the database.
The constructor syntax is: AnaIndex(shared=True, frozen=False, start=None).
The index saves all stations as AnaIndexEntry tuples, in the same order as they come out of the database.
- NetworkIndex
Index for networks, as they come out of the database.
The constructor syntax is: NetworkIndex(shared=True, frozen=False, start=None).
The index saves all networks as NetworkIndexEntry tuples, in the same order as they come out of the database.
- LevelIndex
Index for levels, as they come out of the database
The constructor syntax is: LevelIndex(shared=True, frozen=False, start=None).
The index saves all levels as dballe.Level tuples, in the same order as they come out of the database.
- TimeRangeIndex
Index for time ranges, as they come out of the database.
The constructor syntax is: TimeRangeIndex(shared=True, frozen=False, start=None).
The index saves all time ranges as dballe.TimeRange tuples, in the same order as they come out of the database.
- DateTimeIndex
Index for datetimes, as they come out of the database.
The constructor syntax is: DateTimeIndex(shared=True, frozen=False, start=None).
The index saves all datetime values as datetime.datetime objects, in the same order as they come out of the database.
- IntervalIndex
Index into equally spaced points in time, starting at start, with a point every step time.
Index points are at fixed time intervals, and data is acquired in one point only if it is within a given tolerance from the interval.
The constructor syntax is: IntervalIndex(start, step, tolerance=0, end=None, shared=True, frozen=False).
start is a datetime.datetime object giving the starting time of the time interval of this index.
step is a datetime.timedelta object with the interval between sampling points.
tolerance is a datetime.timedelta object specifying the maximum allowed interval between a datum datetime and the sampling step. If the interval is bigger than the tolerance, the data is discarded.
end is an optional datetime.datetime object giving the ending time of the time interval of the index. If omitted, the index will end at the latest accepted datum coming out of the database.
The data object used by AnaIndex is:
- AnaIndexEntry
AnaIndex entry, with various data about a single station.
- It is a named tuple of 4 values:
- id: station id
- lat: latitude
- lon: longitude
- ident: mobile station identifier, or None
The extraction is done using the dballe.volnd.read function:
- read(cursor, dims, filter=None, checkConflicts=True, attributes=None)
cursor is a dballe.Cursor resulting from a dballe query
dims is the sequence of indexes to use for shaping the data matrixes
filter is an optional filter function that can be used to discard values from the query: if filter is not None, it will be called for every output record and if it returns False, the record will be discarded
checkConflicts tells if we should raise an exception if two values from the database would fill in the same position in the matrix
attributes tells if we should read attributes as well: if it is None, no attributes will be read; if it is True, all attributes will be read; if it is a sequence, then it is the sequence of attributes that should be read.
The result of dballe.volnd.read is a dict mapping output variable names to a dballe.volnd.Data object with the results. All the Data objects share their indexes unless the xxx-Index definitions have been created with shared=False.
This is the dballe.volnd.Data class documentation:
- Data
Container for collecting variable data. It contains the variable data array and the dimension indexes.
If v is a Data object, you can access the tuple with the dimensions as v.dims, and the masked array with the values as v.vals.
The methods of dballe.volnd.Data are:
- Data.append(…)
Collect a new value from the given dballe record.
You need to call finalise() before the values can be used.
- Data.appendAttrs(…)
Collect attributes to append to the record.
You need to call finalise() before the values can be used.
- Data.finalise(…)
- Stop collecting values and create a masked array with all the values collected so far.