pyfrbcatdb package

Submodules

pyfrbcatdb.FRBCat module

description: FRBCat functionality for pyfrbcatdb license: APACHE 2.0 author: Ronald van Haren, NLeSC (r.vanharen@esciencecenter.nl)

class pyfrbcatdb.FRBCat.FRBCat_add(connection, cursor, mapping, event_type)

Bases: object

Class module that adds a decoded VOEvent file to the FRBCat database.

Parameters:
  • connection (psycopg2.extensions.connection) – database connection
  • cursor (psycopg2.extras.DictCursor) – database cursor object
  • mapping (dict) – mapping between database entry and VOEvent value.
  • event_type (str) – type of VOEvent
add_VOEvent_to_FRBCat()

Add a VOEvent to the FRBCat database. This is the main method that iterates over all tables and calls the respective method for each table. Finally, the database changes are committed and the database connection is closed.

add_authors(table, cols, value)

Add author to the database if the ivorn is not in the authors table.

Parameters:
  • table (str) – name of database table
  • cols (list) – cols in database table that need to be added
  • value (list) – values to be set for the cols
add_frbs(table, cols, value)

Add event to the frbs table.

Parameters:
  • table (str) – name of database table
  • cols (list) – cols in database table that need to be added
  • value (list) – values to be set for the cols
add_observations(table, cols, value)

Add event to the observations table.

Parameters:
  • table (str) – name of database table
  • cols (list) – cols in database table that need to be added
  • value (list) – values to be set for the cols
add_radio_measured_params(table, cols, value)

Add event to the radio_measured_params table

Parameters:
  • table (str) – name of database table
  • cols (list) – cols in database table that need to be added
  • value (list) – values to be set for the cols
add_radio_measured_params_notes(table, cols, notes)

Add event to the radio_measured_params_notes table.

Parameters:
  • table (str) – name of database table
  • cols (list) – cols in database table that need to be added
  • notes (list) – list of notes (each note is a string)
add_radio_observations_params(table, cols, value)

Add event to the radio_observations_params table.

Parameters:
  • table (str) – name of database table
  • cols (list) – cols in database table that need to be added
  • value (list) – values to be set for the cols
add_radio_observations_params_notes(table, cols, notes)

Add event to the radio_observations_params_notes table.

Parameters:
  • table (str) – name of database table
  • cols (list) – cols in database table that need to be added
  • notes (list) – list of notes (each note is a string)
check_author_exists(ivorn)

Check if author already exists in the database. If the author is found, set self.author_id to the id found in the database.

Parameters:ivorn (string) – author ivorn from VOEvent file
Returns:boolean if author is found or not
Return type:bool
check_event_exists(ivorn)

Check if event ivorn already exists in database. If the event is found, set self.event_id to the id found in the database.

Parameters:ivorn (string) – event ivorn from VOEvent file
Returns:boolean if event is found or not
Return type:bool
static define_sql_params(cols, value)

Format sql params for the sql command from the cols and values.

Parameters:
  • cols (numpy.ndarray) – cols in the db table the sql command needs to operate on
  • value (numpy.ndarray) – values of the col variables
Returns:

col_sql (formatted cols array as a comma-seperated string), parameters (string of format (%s,%s,…), equal to the number of elements in cols), value (values of the col objects, converted to flattened regular numpy array)
Return type:

str, str, numpy.ndarray
get_authortime()

Get time voevent file was authored from mapping dictionary.

Returns:datetime string of format ‘%Y-%m-%d %H:%M:%S’
Return type:str
get_id_existing(table, cols, value)

Get id of an existing entry in database table.

Parameters:
  • table (str) – name of database table
  • cols (list) – cols in database table that need to be added
  • value (list) – values to be set for the cols
Returns:

id of existing entry in table
Return type:

int
insert_into_database(table, cols, value)

Insert event into the database. This method runs sql command. If not all required parameters are specified, assume this is an update event and return the id for the entry in the table, else return the id of the insert.

Parameters:
  • table (str) – name of database table
  • cols (list) – cols in database table that need to be added
  • value (list) – values to be set for the cols
Returns:

id of insert or id of existing entry in table
Return type:

int
retract(voevent_cited)

Retract event with the ivorn given by voevent_cited. Retracting event should set detected/verified to False in observations table. Database changes are committed and database connection is closed.

Parameters:voevent_cited (str) – event ivorn to be retracted
set_rank()

Return the rank for the event to be inserted. First event of an FRB is rank=1, each next event increments it by 1.

Returns:next_rank, rank of the event to be inserted
Return type:int
update_database(table, cols, value)

If type supersedes we need to update existing table values, else do nothing. This method executes the sql statement.

Parameters:
  • table (str) – name of database table
  • cols (list) – cols in database table that need to be added
  • value (list) – values to be set for the cols
class pyfrbcatdb.FRBCat.FRBCat_create(connection, cursor, frbs_id)

Bases: object

Class module that creates a VOEvent file from the FRBCat database.

Parameters:
  • connection (psycopg2.extensions.connection) – database connection
  • cursor (psycopg2.extras.DictCursor) – database cursor object
  • frbs_id (int) – id in frbs table of FRB to be extracted
add_params(supermapping)

Add radio observations params section to voevent object.

Parameters:supermapping (list) – flattened mapping dictionary in list of dicts
createParamList(params)

Ceate a list of params, so these can be written as group.

Parameters:params (list) – List of params (list of dicts). Each dict contains the information for one parameter.
Returns:paramList, list of lxml.objectify.ObjectifiedElements
Return type:list
create_VOEvent_from_FRBCat()

Create a VOEvent from the FRBCat database. Method gets all information from the database, sets and output name for the VOEvent and calls the create_xml method.

create_xml(xmlname)

Create VOEvent xml file from extracted database values. Calls all the methods to set the different sections and finally calls the save_xml method.

Parameters:xmlname (str) – name of the output file
init_voevent()

Initialize voevent.

save_xml(xmlname, force_pretty_print=False)

Check the validity of the voevent xml file and save as xmlname.

set_author()

Add author section to voevent object.

set_how()

Add How section to voevent object.

set_what()

Add What section to voevent object.

set_wherewhen()

Add WhereWhen section to voevent object.

set_who()

Add who section to voevent object.

set_why()

Add Why section to voevent object.

pyfrbcatdb.FRBCat.parse_mapping()

Read mapping from json file.

Returns:mapping dictionary from mapping.json data file
Return type:dict

pyfrbcatdb.create_VOEvent module

description: Decode VOEvent db entry to xml license: APACHE 2.0 author: Ronald van Haren, NLeSC (r.vanharen@esciencecenter.nl)

pyfrbcatdb.create_VOEvent.create_VOEvent(frb_ids, dbName, dbHost, dbPort, dbUser, dbPassword)

Decode FRBCat entry.

Parameters:
  • frb_ids (list) – list of ids of FRBs in frbs table to extract
  • dbName (str) – database name
  • dbHost (str, NoneType) – database host
  • dbPort (str, NoneType) – database port
  • dbUser (str, NoneType) – database user name
  • dbPassword (str, NoneType) – database user password

pyfrbcatdb.dbase module

description: Common database functionality for pyfrbcatdb license: APACHE 2.0 author: Ronald van Haren, NLeSC (r.vanharen@esciencecenter.nl)

pyfrbcatdb.dbase.closeDBConnection(connection, cursor)

Closes a connection to a DB given the connection and cursor objects.

Parameters:
  • connection (psycopg2.extensions.connection) – database connection
  • cursor (psycopg2.extras.DictCursor) – database cursor object
pyfrbcatdb.dbase.commitToDB(connection, cursor)

Commit changes to database, rollback in case of errors.

Parameters:
  • connection (psycopg2.extensions.connection) – database connection
  • cursor (psycopg2.extras.DictCursor) – database cursor object
pyfrbcatdb.dbase.connectToDB(dbName=None, dbUser=None, dbPassword=None, dbHost=None, dbPort=None, dbCursor=<class 'psycopg2.extras.DictCursor'>)

Connect to a specified PostgreSQL DB and return connection and cursor objects.

Parameters:
  • dbName (str) – database name
  • dbHost (str, NoneType) – database host
  • dbPort (str, NoneType) – database port
  • dbUser (str, NoneType) – database user name
  • dbPassword (str, NoneType) – database user password
Returns:

connection, cursor
Return type:

psycopg2.extensions.connection, psycopg2.extras.DictCursor
pyfrbcatdb.dbase.extract_from_db_sql(cursor, table, column, col, value)

Extract a value from the database.

Parameters:
  • cursor (psycopg2.extras.DictCursor) – database cursor object
  • table (str) – db table name
  • column (str) – db table column name
  • col (str) – db table column name
  • value (str) – value of db table column name to match

pyfrbcatdb.decode_VOEvent module

description: Create a db entry for a VOEvent license: APACHE 2.0 author: Ronald van Haren, NLeSC (r.vanharen@esciencecenter.nl)

class pyfrbcatdb.decode_VOEvent.decode_VOEvent(voevent, dbName, dbHost, dbPort, dbUser, dbPassword, logfile)

Bases: pyfrbcatdb.logger.logger

Class to decode a VOEvent file and insert it into the FRBCat database.

Parameters:
  • voevent (_io.BufferedReader, str) – filestream or filename
  • dbName (str) – database name
  • dbHost (str, NoneType) – database host
  • dbPort (str, NoneType) – database port
  • dbUser (str, NoneType) – database user name
  • dbPassword (str, NoneType) – database user password
  • logfile (str) – name of log file
static get_attrib(v, attribname)

Get xml attributes.

Parameters:
  • v (lxml.objectify.ObjectifiedElement) – VOEvent xml
  • attribname (str) – attribute name
Returns:

v.attrib[attribname]
Return type:

str
static get_coord(v, coordname)
Get coordinate from VOEvent file.
  • transform to HH:MM:SS if coordname=ra
  • transform to DD:HH:SS if coordname=dec
Parameters:
  • v (lxml.objectify.ObjectifiedElement) – VOEvent xml
  • coordname (str) – coordinate name (‘ra’ or ‘dec’)
Returns:

location string in HH:MM:SS.MS for coordname=ra or DD:HH:SS.MS for coordname=dec
Return type:

str
static get_description(v, item)

Return description of parameter.

Parameters:
  • v (lxml.objectify.ObjectifiedElement) – VOEvent xml
  • item (dict) – single dictionary item from mapping
Returns:

Description on parameter is applicable, else None
Return type:

str, NoneType
static get_param(param_data, param_group, param_name)

Get param data for a given attribute.

Parameters:
  • param_data (orderedmultidict.orderedmultidict.omdict) – all param data from VOEvent file
  • param_group (str) – param group in VOEvent which holds param_name
  • param_name (str) – name of parameter to get value for
Returns:

param value if defined in VOEvent, else None
Return type:

str, float, int, NoneType
static get_utc_time_str(v)

Get time in UTC.

Parameters:v (lxml.objectify.ObjectifiedElement) – VOEvent xml
Returns:time as string ‘YYYY-MM-DD HH:MM:SS.MS’
Return type:str
get_value(v, param_data, item, event_type)

Extract the value of item from VOEvent.

Parameters:
  • v (lxml.objectify.ObjectifiedElement, str) – VOEvent xml
  • param_data (orderedmultidict.orderedmultidict.omdict) – all param data from VOEvent file
  • item (dict) – single dictionary item from mapping
  • event_type (tuple) – event type of VOEvent, including citation if applicable, e.g. (‘new’, None)
Returns:

value for item
Return type:

int, float, str, bool, NoneType
parse_VOEvent(voevent, mapping)

Parse VOEvent xml file.

Parameters:
  • voevent (lxml.objectify.ObjectifiedElement, str) – VOEvent xml file
  • mapping (dict) – mapping from mapping.json
Returns:

mapping (mapping from mapping.json with values filled), event_type (event_type and citation if applicable)
Return type:

dict, tuple
process_VOEvent(voevent)

Main method to process the VOEvent.

Parameters:voevent (_io.BufferedReader, str) – filestream or filename
update_FRBCat(mapping, event_type)

Add new FRBCat entry. Calls the FRBCat_add class.

Parameters:
  • mapping (dict) – mapping from mapping.json
  • event_type (tuple) – event_type and citation if applicable

pyfrbcatdb.logger module

description: Logging functionality for pyfrbcatdb license: APACHE 2.0 author: Ronald van Haren, NLeSC (r.vanharen@esciencecenter.nl)

class pyfrbcatdb.logger.logger(filename, DEFAULT_LOG_LEVEL='debug', DEFAULT_LOG_LEVEL_C='warning')

Bases: object

Logger class.

Parameters:
  • filename (str) – log filename
  • DEFAULT_LOG_LEVEL (str) – default log level
  • DEFAULT_LOG_LEVEL_C (str) – default log level to console
start_logging()

Start logging with given filename and level.

pyfrbcatdb.writeCSV module

description: Create a db entry for a VOEvent license: APACHE 2.0 author: Ronald van Haren, NLeSC (r.vanharen@esciencecenter.nl)

class pyfrbcatdb.writeCSV.writeCSV(CSV, dbName, dbHost, dbPort, dbUser, dbPassword, logfile)

Bases: pyfrbcatdb.logger.logger

Class module that write the FRBCat database to a CSV file

Parameters:
  • CSV (str) – CSV filename
  • dbName (str) – database name
  • dbHost (str, NoneType) – database host
  • dbPort (str, NoneType) – database port
  • dbUser (str, NoneType) – database user name
  • dbPassword (str, NoneType) – database user password
  • logfile (str) – name of log file
defineSQLStatement()

Define SQL statement for creating CSV file

writeToCSV()

Dump database to CSV file

Module contents