Importing Data from p:IGI-3 Projects
Overview
With the evolution of the IGI property model import of data from a p:IGI-3 file into p:IGI+ or Metis Transform should involve careful consideration.
Version: 1.7.2+ (Aug 2017)
Usage: Project --> Import data --> Import p:IGI-3 data...
How to use in practice
p:IGI-3 Data Import
With the evolution of the IGI property model, import of data from a p:IGI-3 file requires careful consideration. Each core property in p:IGI-3 has been linked to an equivalent default property, indicator, unit and ratio in the IGI property model, while the User Defined properties are linked using the import systems property name & associated alias linking model.
The assumptions underlying the proposed mapping of default p:IGI-3 properties to the IGI property model, are complex and should be reviewed prior to importing any p:IGI-3 file.
The Import process steps
The import comprises 2 stages followed by an import summary.
- Stage 1 - p:IGI-3 file overview: provides a preview of the first 100 rows of data to ensure the file format and organisation of data is correct. Only properties with data will be displayed and imported from a p:IGI-3 file. At this point the user can select whether to import supported artefacts.
- Stage 2 - Data Linking: a preview of the first 100 rows of data is provided, alongside the property linking table (with the ability to apply/save linking templates and apply bulk edits to property analysis group, indicator and/or unit).
During Stage 1 of the import process the user is also given the option to import Graphs and Palettes from the p:IGI-3 file where possible (p:IGI+ only), however it may not be possible to import some p:IGI-3 artefacts, e.g. if a graph type is not yet supported in p:IGI+ or if user columns are used in the p:IGI-3 artefacts. We advise to just import data and start afresh with Autographs and palettes in p:IGI+.
Stage 2 of the import process maps the p:IGI-3 properties to the IGI property model. Core p:IGI-3 properties are linked using a default mapping while p:IGI-3 user defined properties are linked (where possible) through the automated property name & associated alias linking used across the system.
With respect to a properties units of measures and ratio format all p:IGI-3 properties are imported using their base (not display) units/ratios e.g. p:IGI-3 display = PI, ,[a/(a+b)] vs. base stored value = PI, Euc, a/b. Here the Euc, means the Euclid, or dimensionless unit.
The status of the p:IGI-3 to IGI properties linking is displayed using various status codes used across import, with the addition of a special p:IGI-3 link icon for properties specifically linked using a pre-built p:IGI-3 link:
There are several status levels which prevent import from proceeding. The linking table can be filtered to enable a focused view onto problem links, duplicate links and multiple link options which need resolving prior to import.
To deal with the non-linked problem properties use the Manual Edit Link. Search and select a suitable p:IGI+ property link using the Analyses and Properties search fields and if appropriate define the property Indicator / Unit and Ratio. If the property you wish to link to is not in the p:IGI+ property model there is the opportunity to create a project property in order to capture the data.
Data pertaining to a sample location is duplicated, linking to both respective Sample and Well analysis group properties. Depending on whether samples are attached to a well or not, users might decide to un-assign location data from the relevant analysis group.
Once properties are linked, property Indicators, Units and Ratios, where appropriate, can be managed. On a singular property this can be achieved through the selector columns presented in the data linking window. Where its more appropriate to bulk manage a group of properties e.g. changing the selected property indicator for the gasolines from unknown [un] to height [h] then the Bulk change analysis grous, indicators and units... feature is most appropriate.
The p:IGI-3 property Repeat is by default linked to Parent#.Sample. This property in p:IGI+ is reserved for identifying the parent sample when dealing with picked samples / sub-samples of a whole etc. and so is incorrectly applied in this instance where we have an exact repeat analysis of a sample in order to check the validity of a sample’s analytical value. The p:IGI+ properties Repeat and Link# in the appropriate analysis group metadata should be populated instead. As the Repeat.Analysis group xxx is a flag property it requires pre-processing accordingly in the incoming source file prior to import. The Link#.Analysis group xxx property is a number column and should receive the p:IGI+ database sample row number of the related original sample following import.
With the p:IGI-3 import linking complete it is always advised to save a Linking Template. Import when ready. There is a second chance to save Linking templates on the Import summary screen.
Following import the user is presented with an Import summary comprised of several sections:
- Project Data Population Summary - Details the population and coverage of data across the samples and the properties linked during import.
- Datum Summary - To assist with the assignment of true vertical depth (TVD) values to project samples based on present measured depth (MD) values all wells and individual samples must have a depth datum specified. This can be set in bulk here. By default Kelly Bushing is selected.
- Activity Summary - A detail of any errors thrown during import. In this file most errors concern the inability to import p:IGI-3 graphs and palettes into p:IGI+ (if choosen to be imported).
- Well Merge Summary - During import, sample well data from a common well is merged together to populate a single associated well artefact. Any conflict and resolving action taken during the well merge is presented.
Once content select Finish. It is at this point that Undo/Redo Ctrl+Z/Ctrl+Y can be used to remove the imported data from the project as one bulk action.
p:IGI-3 Import Discussion
One of the downfalls of the import approach described above, i.e. p:IGI-3 to p:IGI+ default linking mapping model, is that it does not utilise p:IGI+’s new data structure (a property naming convention providing clear differentiation on property name, used analytical work flow and associated other property metadata) to the full. As seen the data is imported maintaining the p:IGI-3 data model structure and its associated reservations:
- Molecular/gas sample data spread over multiple rows
- A limited / narrow standard property model
- An inability to cross plot properties against one another during quality control / interpretation
The image above shows an example p:IGI-3 import into p:IGI+ relating to medium resolution molecular data. A) Is the resultant allocation of data in p:IGI+ using the p:IGI-3 import pipeline. This approach maintains the spread of sample data across multiple rows, with the need for associated metadata to capture the GC-fraction and reporting indicator B) Is the appropriate layout of molecular data in p:IGI+ preserving the data'd analytical origin, while also storing efficiently the multiple reported indicators.
In certain situations the direct import of a p:IGI-3 file to p:IGI+ may be appropriate (in this case it is advised to use both the Default property link mapping and the Unknown indicators where appropriate, while accompanying the data with analytical source property metadata (Fig above, A). But where a p:IGI-3 project contains a mix of data from multiple wells / sources across a medium/large region it is advised that a direct p:IGI-3 import to p:IGI+ is not a suitable course of action (Fig above B).
Instead the segregation of a p:IGI-3 project into appropriate parts based on the analysis technique used, indicator recorded etc. using p:IGI-3 SampleSets, Custom Pages (only columns with data) and data export (not covered here) is strongly advised prior to import via text files.
Video tutorials
None Available