Improvements

The improvements in pdrLoader version 7.5.0 release include some enhancements and defect fixes from the previous release.

Functional enhancements version 7.5.0

The functional enhancements in pdrLoader v7.5.0 include:

  • Configuration structure and feature updates to support containerisation
    • Now supports parsing of system and environment variables into properties file
    • Now supports data vault configuration (docker secrets file) and parsing of this configuration into properties file
  • Now supports loading of flat CSV files without column headings, the column headings are implied by the database table
  • Now supports flexible filtering of rows to be loaded, new table PDR_DATA_FILTER_CONFIG >
  • Now logs changes to mapping configuration in PDR_* management tables to the application log
  • Now attempts to roll forward previous version configuration in the event that missing configuration is detected in PDR_REPORT_RECORDS
  • pdrEnvironment scripts now use wildcard for CLASSPATH for third party libraries
  • Now supports Azure Key Vault, AWS Secrets Manager and Google Secrets Manager integration
  • Added wrapper scripts for data export facility
  • New checks for correct setup of PDR_* database management tables at start up
  • Now ensures that any rows within PDR_REPORT_RECORDS with a destination table of <NEW> are set as inactive.
  • Now supports 7z, tgz & gz as compression
  • Included MANFCAS, NORFCAS as data sources for production
  • New thread parameter file_load_thread_{N}_mask_mode to allow EXCLUSIVE & AND options for include/exclude file mask specification
  • New command line script pdrConnectionTest to validate database connection configuration
  • Update SQL Server JDBC driver mssql-jdbc-10.2.0.jre11.jar
  • Update Oracle Database 21c (21.4.0.0.1) JDBC Driver
  • Added API to report file load errors
  • Change default threshold for maximum archive requests and rejections from 10 to 5. This is to ensure that the automated recovery process does not consume the 10 re-delivery attempts from the AEMO end before manual intervention is possible.
  • Add ability to manage maximum request count within manifest processes
  • Now uses file_load_threads_active, replacing file_load_threads

Bug fixes version 7.5.0

The bug fixes in pdrLoader v7.5.0 include:

  • Log4j2 vulnerability fix
  • Correction to define FILE_ID for gas INT format files. This is required to support reporting views in pdrMonitor
  • Now removes files from Requests folder when timeout detected