TAR-SEG-D For NDR Reporting Of Non-Compliant SEG-D Data

This option to report non compliant SEG-D files to the NDR requires concatenating individual files on a spatial basis relating to their acquisition methodology. For example, all the SEG-D files in a single acquisition line (sail-line) would be spatially sequentially concatenated using the TAR command to output a single gzipped TAR file (TAR-SEG-D file). 

While all the SEG-D files per acquisition line can be combined into a single TAR-SEG-D file, if the output TAR-SEG-D size exceeds 400 Gigabytes then multiple TAR-SEG-D files should be made per acquisition line.

This reporting option is applicable for tape native SEG-D versions or single shot SEG-D files. The aim of the TAR-SEG-D process is to reduce a very large number of small files that can be an unmanageable solution for many NDR end users into a “reasonable” number of files for the NDR end user. The NDR end user is then able to select spatially related data (e.g. sail-line) to be delivered where the files are grouped together by a spatial connection, and the file size is up to ~ 400’s GBytes.

TAR commands are available free on all operating systems, TAR is an open published format and is commonly used for backup solutions.

Each TAR-SEG-D file also requires its own supporting file to be created to allow quick preview of the contents in the NDR and to QC the contents of the TAR-SEG-D file. A project containing TAR-SEG-D files must also have two project supporting files present. The supporting files and the TAR-SEG-D file must follow the naming convention defined below, where spaces are not permitted in any filenames.

1. filename.segd.tar.gz - a gzipped TAR file made from multiple SEG-D files. 
2. filename_segd_listing.txt file - For each TAR-SEG-D file there is a matching SEG-D listing file, the naming must match the filename with the “.segd.tar.gz” in the end of the TAR-SEG-D file replaced with “_segd_listing.txt”.
3. surveyname_field_data_project_line_listing.txt - one required for each survey - A line listing text file containing the spatially related acquisition ordering, that corresponds to the field data files being uploaded to the NDR, e.g. sail-line. 
4. surveyname_field_data_project_info.txt - one required for each survey, this metadata file detailing the SEG-D format used in the survey and range of field file index (ffid) in each TAR-SEG-D file. 

Examples of these file names for a survey called TT133D0002 would be:

1. TT133D0002_TOB13-0990P1-033.segd
2. TT133D0002_TOB13-0990P1-033_segd_listing.txt
3. TT133D0002_field_data_project_line_listing.txt
4. TT133D0002_field_data_project_info.txt

All of these files must be present to upload the TAR-SEG-D dataset. The definitions of the files are detailed in the NSTA form and manner document.

Workflow to create TAR-SEG-D file for an acquisition line 

The flow diagram shows the stages of creating the TAR-SEG-D dataset. The definitions of the  files are detailed in the form and manner document and the file creation stages are detailed below:

1. If working from tape media extract SEG-D files to disk with suitable tape control code, the suggestion is to put files per acquisition line per directory
2. Extract SEG-D header metadata either with SEG-D reading software and create a surveyname_field_data_project_info.txt file. Or after creating a TAR-SEG-D file preview it in the NDR upload interface. This will create a template for the surveyname_field_data_project_info.txt and populate the first two sections of the file. 
3. Read each SEG-D file for the acquisition line and extract the FFID for each file, this can be done with either SEG-D reading software or a Python code like “scansegdhdr.py”  which is available. The list of files and FFID are then written to the filename_segd_listing.txt file.
4. Create a compressed TAR file from the selected SEG-D files, this can be done with your system-suitable TAR command, e.g. 

tar --numeric-owner -cvzf filename.segd.tar.gz linename*.sgd 

Or on Linux / MacOS using pigz software to run the compression in parallel:

tar --use-compress-program="pigz -k --best" --numeric-owner -cvf linename.segd.tar.gz linename*.sgd 

5. Readback the TAR-SEG-D file to ensure that all the files were written to the file and appended to the filename_segd_listing.txt file, during upload this will be cross-checked to ensure all the expected files were present on the readback.

tar --full-time -tvf filename.segd.tar.gz

6. Create a text file surveyname_field_data_project_line_listing.txt containing the spatially related acquisition ordering, that corresponds to the field data files being uploaded to the NDR, e.g. sail-lines.

Workflow for uploading a TAR-SEG-D dataset

Once the TAR-SEG-D files and supporting files have been made they can be uploaded using the Browser based upload page in the NDR. Once the TAR-SEG-D file and its supporting files have been put into an upload folder, the folder can be selected for upload. Before the TAR-SEG-D file is made available to be selected to upload, all of the required CTAGs must be present to upload the TAR-SEG-D dataset and have to be assigned CTAGs in a defined order. The order is shown below and prior CTAGs must be set before upload is allowed for each of the successive datasets.

1. Acquisition report - ctag "REPORT_ACQUISITION"
2. Observers logs - ctag "OBSERVER_LOG"
3. Source signature - ctag "SOURCE_SIGNATURE"
4. Final Navigation data - ctag "FINAL_NAVIGATION" (excluding SEG-Y Nav-seis)
5. Field data - ctag "DATA_ACQUIRED"

Each CTAG requires the previous ones to have been set before it becomes available in the CTAG setting popup box. Therefore you cannot see or set a CTAG for  "SOURCE_SIGNATURE" until "OBSERVER_LOG" and "REPORT_ACQUISITION" have been set on documents either in the current session, already uploaded or in other sessions (in the same browser type) which have started the upload for the files with those CTAGs assigned.

Individual files for each CTAG are not required, some of these CTAGs may be in the same physical file. Multiple CTAGs can be set for the relevant dataset against individual files. For example, the source signature might be included within the acquisition report, in which case the acquisition report should be marked with at least two CTAGs, Report_Acquisition and Source_Signature.

The flow diagram shows the different files and CTAGs required to allow uploading to the NDR. In this example workflow, CTAGS are only manually assigned to the acquisition report pdf file as the file type and naming of the other files allows the automatic assignment of CTAGS.

Software tools required to create TAR-SEG-D file and supporting files