We’re pleased to advise that a new version of the NDR User Interface, v20240712_13, has been deployed and is available to all users.
In this NDR Update we summarise:
+ Operations moved to MS Azure v3
+ New map functionality and layers for Seismic Surveys
+ File Format metadata corrections
+ National Hydrocarbon Data Archive migration to the NDR
+ Continued migration of offline archive data to the NDR
Please read on for further details on our recent developments.
Best Regards,
Andy Thompson
NDR operating on MS Azure version 3
The NDR is principally hosted from the NSTA’s presence in MS Azure (with data storage and some services operated from AWS). In 2023 Microsoft announced their intention to retire version 2 of the Azure Cosmos DB .NET SDK by the end of August 2024, the NDR would need to be ready to operate on version 3 ahead of that time. In addition, Microsoft has also retired components of the Azure batch service and an API Management service.
Somewhat like upgrading a software application to a version that is compatible with a new operating system, Osokey’s development team needed to update, test, certify and deploy some 25 Azure functions that contribute to the operation of the NDR user interface and related services.
Planning began in December of last year, with updates being completed in batches by the Osokey team over two months, early in 2024. Taking a staged approach meant that the updates could progress while avoiding disruptions to the NDR service. The Osokey team successfully completed the upgrades by rebuilding Batch services and updating Osokey’s own code, such that the NDR now runs entirely on the new Azure Batch Service and API Management service.
This was a significant exercise for Osokey’s development team and, having been delivered to a typically high standard, nobody noticed. We were ready to issue a notification to users to expect a short period of downtime while the overall service was restored however, that wasn’t needed, thanks to the great work by the team.
In fact, we’ve recently passed 3 years since the Osokey NDR service was launched – no outages to date, scheduled or otherwise. That’s an endorsement of the move to a cloud-based service and a service provider that is focused on service excellence.
NDR Map enhancements
User-defined spatial selection from the NDR Map
The ability to select information of interest using spatial constraints has been present in the NDR Map since the current service was launched; original functionality enabled the user to focus on a specific Quadrant, Block or Field, depending on the zoom level. Clicking on a given quad will filter the Projects table to show only the Well Projects that lie within that Quad, and the Seis and other Projects that cross into/over that Quad. The same is true for reference blocks and for the perimeters of hydrocarbon fields. We subsequently introduced various active layers that produce similar results based on the boundaries of offshore wind developments and carbon storage permits, based on information published by The Crown Estate and Crown Estate Scotland.
In our latest release we have introduced additional and much requested control over spatial selection. Clicking this icon , which is found towards the top left corner of the map, enables the user to define their own polygon. Closing the polygon with a double click with filter the Projects table to show the wells, surveys etc. that sit within the user-defined shape (there is a buffer, so there may be a few items included that are outside, but close to the polygon).
New layers for seismic surveys
Another request from users was to better display the availability of seismic data on the map, and to enable data found via the map to be loaded to the Project and Files tables.
Three new layers have been added to the default Layers Menu:
+ 3D+ Seismic Projects (3D, 4D and OBS Survey Outlines)
+ 2D Seismic Projects (2D Seismics Survey Outlines)
+ Site Seismic Projects (Site Survey Outlines)
Each of these has four sub-layers:
+ Post-Stack Data - Orange polygons relating to seismic projects that have available Post-Stack SEG-Y data
+ Pre-Stack Data Green polygons relating to seismic projects that have available Pre-Stack SEGY data
+ Acquisition Data Blue polygons relating to seismic projects that have available SEG-D field data or SEG-Y nav-merge data
+ No Seismic Data Black polygons relating to seismic projects that have no available SEG-Y or SEG-D data
Clicking on a project polygon filters the Project table to show the selected project and launches a pop-up with summary information for the project, including a View Files button. Clicking View Files will populate the Files Table with the files associated to the survey, which can then be previewed (both from the preview button and the right mouse menu), downloaded or added to a Session Basket.
If seismic data files are populated into the table, the file name will be visible on the map. Clicking the on the file name will launch a separate pop-up, from which the user can either launch a preview of the SEG-Y data or the SEG-Y Headers. These previews will open in separate browser tabs (read about this new functionality in the Seismic Data Quality Tool section below).
It is often the case that several project polygons will overlap, where successive surveys have been acquired. The user can click successively over the same space to cycle through projects that occupy the same space on the map. This is also available to cycle through files that have been posted to the map.
Equivalent layers are available for 2D surveys and Site Surveys (although we acknowledge the 2D polygons are perhaps less useful than the 3D+ and Site Survey layers – we’ll continue to work on these).
Please refer to our Support Centre for more information on these new functions of the NDR Map.
https://support.uk-ndr.co.uk/hc/en-gb/articles/4403134130962-Map-Functionality
https://support.uk-ndr.co.uk/hc/en-gb/articles/20159008923154-Map-Seismic-Layers
File Format metadata corrections.
File Format metadata has been reviewed for close to 900,000 items held in the NDR data collection.
Of those, approaching 99,000 files have had the File Format metadata updated – split between those that have been Corrected (~54,000 items) and those that have been Standardised (~44,000 items).
- Corrected - where the value described the file as being one format when it is actually another (for example, the file was labelled as PDF when it is actually TIFF)
- Standardised - where the assigned value was correct, but to a variant spelling/expression of the term we use in the NDR (e.g. the label was SEG-Y, when the current system uses SEGY)
The remaining 800,000 files had values that satisfied our initial review, where the File Format label appears to be a valid representation of the actual file format.
There are around 20 file formats that we would expect data to be saved to, prior to being loaded to the NDR. Each type of information is to be provided in a specified format. In a few cases the information can be provided in more than one format.
Licensee are required to provide information in the correct “exchange format” to be compliant with the NSTA’s requirements, meaning that, should it subsequently be published, it will be readily accessible and reusable by future users of information.
Today, data that is not correctly formatted cannot be reported via the NDR. Thanks to the quality control measures in place in the NDR, we can be confident that the format of data is known before it is uploaded. Since we transferred to the current NDR service, provided by Osokey, in 2021 the upload procedure has automatically applied File Format metadata when a licensee uploads a file.
The NDR has been operational since 2019; the information held in the first version was migrated to the current system in 2021. Most of the original NDR data collection was migrated from a preceding industry solution, together around three quarters of the data we hold today, by file count.
Having a value for File Format that accurately describes the file benefits licensees looking to improve the classification of their previously reported information. Around one half of the 54,000 + corrections to File Format have been from invalid to valid values, including 2,012 JPEG files, over half of which are Core Images, for which JPEG is a compliant file format.
Users should know the format of data before they download it to their systems. Today the NDR doesn’t permit information to be uploaded in Zip files, however many such files were uploaded to legacy systems and mislabelled as another format. We have identified and corrected almost 3,800 Zip files that we’re previously labelled as one of 17 other file formats.
Over 24,000 of the changes we’ve made to incorrect labels have been to file formats that are not currently and have never been compatible with the requirements for reportable information.
Having a more accurate description of each item in the data collection means that those who are looking for certain data will have an improved likelihood of finding what they are looking for. Users should expect further improvements to other aspects of file metadata as we continue to improve the standard and quality of data that is available from the NDR.
Seismic Data Quality Tool
Osokey’s development team has augmented the NDR with the addition of truly powerful insight into seismic data. Added earlier this year, the SEG-Y Trace Header Display Tool enables users to query NDR seismic data in the cloud directly from their browser and the NDR User Interface (UI). Not only this, but users can also interrogate files on their own network, prior to uploading them to the NDR.
The tool makes it possible to view the various constituent parts of SEG-Y files, including the textual (EBCDIC), binary and trace headers, and trace preview for local files (which has always been available for NDR cloud stored files).
In the UI, the preview is launched from the right mouse context menu in the files table, opening in a new browser tab. Alternatively, the “Local SEG-Y preview” option is available under the “Data submission” drop-down menu (available to users with the Company Data manager role).
Header information can be plotted in a variety of 2D and 3D plot styles, providing the ability to QC content for consistency and completeness. In addition to the established vertical profile trace display option, users can also generate a ‘timeslice’ image of seismic amplitudes at a specified time or depth value. All plots can be captured as images to use outside the NDR, be that in a document or presentation, or adding to an email to our Support Team to illustrate an issue etc. Please remember to add the NSTA’s attribution statement to acknowledge the source of any images reused from the NDR – this is available from the NDR User Agreement that all NDR users must agree to before entering the system. https://ndr.nstauthority.co.uk/terms/NDR_TermsAndConditions_2023_07_01.pdf
We’re expecting the SEG-Y Trace Header Display Tool to add significant value to users of geophysical data, whether that is to enable more in-depth scrutiny of online data before deciding to download it, troubleshooting issues with data that has been obtained from the NDR, investigating issues with data that cannot be loaded due to data quality issues or otherwise. Please take a look at the support materials below, make use of this functionality to gain a full understanding of the data available in the NDR, and do let us know how you get on.
https://support.uk-ndr.co.uk/hc/en-gb/articles/18049026013330-SEG-Y-Trace-Header-Display-Tool
The NSTA recently announced the successful completion of our collaboration with the British Geological Survey (BGS) to migrate offshore petroleum licence information from the BGS held National Hydrocarbon Data Archive (NHDA) to the NDR. https://www.nstauthority.co.uk/news-publications/north-sea-data-upload-will-support-energy-transition/
The NHDA was set up in the early 2000’s as a joint initiative between the NSTA’s predecessors, BGS and industry to facilitate the secure archival of licence data, while achieving relief of the obligation to retain data in perpetuity on the part of the licensee.
Seismic and well data that was archived in the NHDA was analysed versus the content of the NDR. Certain information was already available via the NDR; of the remaining NHDA some 4,300+ files were migrated from tape and hard drives to the NDR cloud storage.
The migrated NHDA data is of a vintage that means it has all been automatically marked as released in the NDR. Through this collaboration we’ve made archived data immediately accessible to all NDR users.
Offline archive data to the NDR
When the NDR was established in 2019 the NSTA inherited both the online data collection and an offline archive of data that had previously been submitted to the NDR’s forerunner, UKOilandGasData.
Most of the offline archive is held on magnetic tape, a commonly used medium for storage and exchange of seismic data in particular. A lesser proportion of media includes hard drives and even floppy disks (for those of us who remember using those!). The majority of offline data relates to seismic surveys. The original NDR service hosted post-stack seismic data online, whereas pre-stack and original ‘field’ data was held on a near-line server. All of that data had been submitted, in duplicate. to the service provider on media. The duplicated media was held in geographically separate locations, establishing a resilient archive.
When we moved to the current service in 2021 the online collection was migrated directly to the cloud. Since then, we’ve been systematically working through the offline archive to identify whether data held on media has already been uploaded. Anything that has not been uploaded, and that qualifies as ‘reportable information’ has been identified and prioritised for upload to the NDR cloud storage, meaning licensees won't be asked to repeat the work of submitting that data to the NSTA, and that potential end user will be better able to find available data and download it to put it to use.
In recent years a good deal of the offline archive has been migrated to secure cloud storage. This includes 3D seismic survey field data that was selectively processed in 2022, in support of the NSTA’s 1st Carbon Storage Licensing Round, and subsequent project work to upload most of the remaining offline 3D seismic data.
This year we’ve been able to move ahead with the conditioning and upload of 2D seismic data. Working with service partners Moveout Data, data for 11 surveys relating to the Southern North Sea and Celtic Sea/English Channel areas was successfully processed.
We’re currently working with Moveout to process data for a further 38 surveys, with the aim being to securely migrate this valuable source of legacy information as soon as practicable.
Comments
0 comments
Article is closed for comments.