Metis Project Merge

Usage:  Metis Tools --> Data Merge and Deduplication...

How to use in practice

The project merge is one of Metis Transforms’ unique tools. It allows you to group together well information and sample information to create unique wells and samples, and assign samples to wells in the system. It is a requirement to undertake a project merge prior to uploading any data to Metis Staging. The merge step provides internal identity to the samples in Metis Transform.

When a row is first imported into Metis Transform we consider it both a candidate sample and a candidate well. The term candidate is used to indicate that this might become, or contribute toward, an actual sample or well in the system. You can think of these as being ‘rows in a spreadsheet’ – they have no real sense of being a sample or belonging to a well until we run the project merge.

In the project merge we bring together one or more candidate rows to create project wells and project samples. Project wells are shown as well artefacts, and project samples are just samples in the spreadsheet. You can tell whether a sample is a candidate sample by inspecting the Sam status.Metis property - an empty value implies a candidate sample, but if the string "Metis sample" is shown you are looking at a project sample.

Only project wells and project samples will be considered for uploading to the Staging database.

The Transform project merge has two stages, first Transform merges wells before moving onto samples.

To launch the merge wizard select Metis Tools -> Data Merge and Deduplication … from the menu. 

Well merge and identification

Once the project merge wizard is launched, at Stage 1, the user is prompted to select properties to be used as well merge criteria. For this first stage there's a restriction to just using well properties.

Merge criteria should be selected based on how they group a dataset into distinct entities. In most cases we are only likely to have Well name.Well available, although sometimes it helps to add Lat.Well, Long.Well, Country.Well, UWI.Well and potentially other properties to ensure we only merge together wells we wish to.

Stage 2 shows you the results of the merge, grouped by the selected criteria. Most wells will hopefully merge without conflict, as shown by the green tick next to the well name, however if you have conflicting data (for example the same well is given different Drillers Reference height values) this will be indicated by the presence of a red cross, and the conflicting property will be shown, with a checkbox allowing the user to specify which value to retain. The number in square brackets tells you the number of rows merging together. You can review all the values (and their latest source) in the right-hand panel, which shows the properties, the source of the data and the merge action to be selected.

Merging the wells in the project is a subtle process. It is really doing two things. First it brings together all the information about wells in the project to merge the information, but secondly it also assigns samples to the created wells.

When running the merge process, the user interface forces you to first merge the wells together and then merge the samples. If you run the merge process again it will remember your choice of criteria allowing you to quickly progress through the well merge stage. It will often to be necessary to run several sample merges with different merge criteria, but typically only one well merge is required.

If the results of the well (or sample) merge are not as anticipated it is possible to Refine Well Identification Criteria and see if a better grouping can be found. It is also important to check for missing rows by reviewing the number of rows with no merge criteria.

To progress to the sample merge stage select Confirm and continue to sample merge.

Sample merge

Sample data is often provided in spreadsheets delivered from labs, sometimes all one lab, sometimes from a range of labs, which can arrive at different times, and typically will not group together all the analyses from one sample on one row. To obtain the full multivariate picture of our sample characteristics we need to merge this information onto a single row. We need to be confident we are not bringing together information from different samples however - this could be very misleading!

Again for the sample merge we can choose many merge criteria. You almost always want to consider the well name / UWI as one of the elements in the merge criteria for samples (you never want to bring together information from samples in different wells). It is also often useful to add the sample depth as a merge criterion, either Base MD.Sample or Top MD.Sample. It can also be helpful to include information on the sample type (e.g. Category.Sample, Rock type.Sample) and on the sample lithology (e.g. Lith.Geol). 

If you are lucky and all your samples have unique, well defined identifiers, then you might be able to use Name.Sample (or another identity property, e.g. Internal#.Sample) to merge information easily.

Once you have chosen the merge criteria, select Merge Samples to group the rows.  When samples merge without any conflicts you will see green ticks. It is still necessary to review the merges and ensure they are intentional. The number of candidate / project samples that are combined in each group to create the project samples is shown inside the square brackets. You can review and inspect each group, and within these, each analysis. If any data conflicts in any analysis this will be highlighted with a red cross. Conflicts must be resolved for a sample to merge successfully.