Introduction¶
This manual describes all GBDS relational databases tables, schemas, and information. GBDS uses a relational database to store the metadata about people, transactions, notification groups, setting overrides, and exceptions.
The document is divided into four sections that describe:
- The General tables;
- The Unsolved Latent (UL) tables;
- The Notification tables;
- The GBDS Settings tables.
General tables¶
The general tables are tables that retain vital information for GBDS operation, as exceptions, people information, transaction information, and others. Those tables are described below.
gbds.people¶
The people table is intended to store the indexes for all people stored in GBDS database,
and is described as follows:
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| id | bigint | 20 | not null | Primary key; Relational database private key |
| pguid | varchar | 255 | not null | HBase stored person UGUID |
| deleted | tinyint | 1 | null | Indicates whether the candidate was deleted to be excluded when performing candidates listing operations |
gbds.people_version¶
The people_version table is intended to store the information about the last changes in a
person’s register, and is described as follows:
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| version | int | 11 | not null | Primary key; Version index (incremental from the first change) |
| person_id | bigint | 20 | not null | Reference to person’s gbds.people.id |
| _timestamp | datetime | 6 | null | Timestamp for the current version of a person |
| deleted | tinyint | 1 | null | Indicates whether the candidate was deleted to be excluded when performing candidates listing operations |
| active | tinyint | 1 | null | Defines if a version of a people is elegible for Master Record transactions |
gbds.transactions_ref¶
The transactions table is intended to store the indexes for all transactions stored in GBDS
database, and is described as follows:
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| id | bigint | 20 | not null | Primary key; Relational database private key |
| tguid | varchar | 255 | not null | HBase stored transaction UGUID |
gbds.transactions¶
The transactions table is intended to store information about quality analysis of the
biometric data for a transaction.
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
id
|
bigint
|
20
|
not null
|
Primary key;
Relational database private key
|
tguid
|
varchar
|
255
|
not null
|
Reference to transaction’s
transactions_ref.tguid |
| pguid | varchar | 255 | null | HBase stored person UGUID |
active
|
tinyint
|
1
|
not null
|
Defines whether a version of a person is
elegible for Master Record transactions
|
finger_quality_extracted
|
tinyint
|
1
|
null
|
False if the fingerprints can be extracted
|
face_quality_extracted
|
tinyint
|
1
|
null
|
False if the face can be extracted in the
background. True if already extracted
|
| created | datetime | 6 | null | Transaction creation timestamp |
| updated | datetime | 6 | null | Transaction last modification timestamp |
enroll_status
|
varchar
|
255
|
null
|
Status of the enroll (as
ENQUEUED,PROCESSING, FAILED, and others) |
quality_status
|
varchar
|
255
|
null
|
Status of the quality analysis (as
OK,PENDING, APPROVED, and others) |
| extraction_time | varchar | 255 | null | Elapsed time for template extractions |
| extraction_quality | varchar | 255 | null | Elapsed time for quality extractions |
| match_time | int | 11 | null | Elapsed match time for this transaction |
| total_time | int | 11 | null | Total operation minus queue waiting time |
type
|
varchar
|
20
|
null
|
Transaction type. Enum:
ENROLL,UPDATE, VERIFY, IDENTIFY |
| fingerprint_global_quality | int | 11 | null | Fingerprint global quality score |
| global_quality | int | 11 | null | Profile global quality score |
quality_extraction_api_id
|
varchar
|
255
|
null
|
API id that will perform or performed
quality extraction
|
latent
|
tinyint
|
1
|
null
|
Flags whether the transaction is a latent
search
|
ul
|
tinyint
|
1
|
null
|
Flags whether the transaction is a UL
search
|
api_id
|
varchar
|
255
|
null
|
The ID of the API instance that received
the transaction
|
num_fingers
|
int
|
11
|
null
|
The number of fingerprints in the
transaction
|
num_faces
|
int
|
11
|
null
|
The number of face images in the
transaction (0 or 1)
|
| gbds_version | varchar | 255 | not null | GBDS version that processed the transaction |
gbds.transaction_fields¶
The transaction_fields table is intended to store the information about the quality information
and link it with the non-biometric information of the fields table.
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| id | bigint | 20 | not null | Primary key; Relational database private key |
| fkey | varchar | 255 | null | Field key describing the contained information (e.g. whether it is a blood type, mother’s name, birth date, document, etc) Reference to |
| type | varchar | 255 | null | Type of the information (search
or Reference to |
| fvalue | varchar | 255 | null | Field effective value (e.g. the document number) Reference to |
| transaction_id | bigint | 20 | not null | Id of the transaction. Reference to |
gbds.transaction_fingerprint_quality¶
| Column | Type | Size | Value |
|---|---|---|---|
| transaction_id | bigint | 20 | not null |
| idx | int | 11 | not null |
| image_quality | int | 11 | null |
| template_quality | int | 11 | null |
| minutiae_count | int | 11 | null |
| blank | varchar | 100 | null |
| contrast | varchar | 100 | null |
| fingerArea | varchar | 100 | null |
| fingerCenterX | varchar | 100 | null |
| fingerCenterY | varchar | 100 | null |
| fingerExtentX | varchar | 100 | null |
| fingerExtentY | varchar | 100 | null |
| hasjoint | varchar | 100 | null |
| __index | varchar | 100 | null |
| marginBiteEast | varchar | 100 | null |
| marginBiteNorth | varchar | 100 | null |
| marginBiteSouth | varchar | 100 | null |
| marginBiteWest | varchar | 100 | null |
| nfiq | varchar | 100 | null |
| orientation | varchar | 100 | null |
| sizeX | varchar | 100 | null |
| sizeY | varchar | 100 | null |
gbds.transaction_face_quality¶
| Column | Type | Size | Value |
|---|---|---|---|
| transaction_id | bigint | 20 | not null |
| idx | int | 11 | not null |
| image_quality | int | 11 | null |
| template_quality | int | 11 | null |
| autoBrightness | varchar | 100 | null |
| bgBelowPictureQuality | varchar | 100 | null |
| bgBlueStandardDeviation | varchar | 100 | null |
| bgDarknessQuality | varchar | 100 | null |
| bgGreenStandardDeviation | varchar | 100 | null |
| bgRedStandardDeviation | varchar | 100 | null |
| bgUniformityQuality | varchar | 100 | null |
| blurCompliance | varchar | 100 | null |
| busyBackground | varchar | 100 | null |
| busyBackgroundInCropped | varchar | 100 | null |
| cropContainmentError | varchar | 100 | null |
| eyesLookDown | varchar | 100 | null |
| eyesLookLeft | varchar | 100 | null |
| eyesLookRight | varchar | 100 | null |
| eyesLookUp | varchar | 100 | null |
| eyesObstruction | varchar | 100 | null |
| eyesTooClosed | varchar | 100 | null |
| eyesTooOpen | varchar | 100 | null |
| faceDown | varchar | 100 | null |
| faceLeft | varchar | 100 | null |
| faceObstruction | varchar | 100 | null |
| faceOrientationPitchCompliance | varchar | 100 | null |
| faceOrientationRollAngle | varchar | 100 | null |
| faceOrientationYawCompliance | varchar | 100 | null |
| faceRight | varchar | 100 | null |
| faceUp | varchar | 100 | null |
| glasses | varchar | 100 | null |
| grayscaleSpan | varchar | 100 | null |
| hat | varchar | 100 | null |
| heavyGlasses | varchar | 100 | null |
| icaoCompliance | varchar | 100 | null |
| leftEyeX | varchar | 100 | null |
| leftEyeY | varchar | 100 | null |
| mouthObstruction | varchar | 100 | null |
| mouthOpen | varchar | 100 | null |
| obstruction | varchar | 100 | null |
| openMouth | varchar | 100 | null |
| pictureHeight | varchar | 100 | null |
| pictureWidth | varchar | 100 | null |
| pixelated | varchar | 100 | null |
| redEye | varchar | 100 | null |
| result | varchar | 100 | null |
| rightEyeX | varchar | 100 | null |
| rightEyeY | varchar | 100 | null |
| saturated | varchar | 100 | null |
| saturationGrayscaleDistribGrade | varchar | 100 | null |
| saturationNumGrayTones | varchar | 100 | null |
| saturationOverExposure | varchar | 100 | null |
| shadows | varchar | 100 | null |
| skinColorCompliance | varchar | 100 | null |
| smile | varchar | 100 | null |
| smilingMouth | varchar | 100 | null |
| spoof | varchar | 100 | null |
| spoofGrade | varchar | 100 | null |
| tiltAngle | varchar | 100 | null |
| tooDark | varchar | 100 | null |
| unnaturalSkinColor | varchar | 100 | null |
| visibleTeeth | varchar | 100 | null |
| wrongFacePose | varchar | 100 | null |
| wrongShoulderPoseLeft | varchar | 100 | null |
| wrongShoulderPoseRight | varchar | 100 | null |
gbds.biometrics¶
The biometrics table is intended to store the biometrics of a person, being linked
to the transaction and the people_version tables. It is described as follows:
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| id | bigint | 20 | not null | Primary key; Relational database private key |
| idx | int | 11 | null | Index of the current biometric |
| quality | int | 11 | null | Quality score for the extracted biometric template |
| type | varchar | 255 | null | Modality of the current biometric |
| person_id | bigint | 20 | not null | Reference to person’s gbds.people.id |
| transaction_id | bigint | 20 | not null | Reference to transaction’s
transactions_ref.id |
| person_version | bigint | 11 | not null | Reference to person’s
people_version.version |
gbds.fields¶
The fields table is intended to store the non-biometric information of a person, being
linked to the transaction and the people_version tables. It is described as follows:
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| id | bigint | 20 | not null | Primary key; Relational database private key |
| fkey | varchar | 255 | null | Field key describing the contained information (e.g. whether it is a blood type, mother’s name, birth date, document, etc) |
| type | varchar | 255 | null | Type of the information (search
or label) |
| fvalue | varchar | 255 | null | Field effective value (e.g. the document number) |
| person_id | bigint | 20 | not null | Reference to person’s
person.id |
| person_version | int | 11 | not null | Reference to person’s
people_version.version |
gbds.exceptions¶
The exceptions table is intended to store the information of any biometric exception and
its treatment. It is described as follows:
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| id | bigint | 20 | not null | Primary key; Relational database private key |
| aguid | varchar | 255 | not null | HBase stored exception UGUID |
| comments | varchar | 255 | null | Any comments given when treating the exception |
| _timestamp | datetime | 6 | null | Timestamp for the creation of the exception |
| status | varchar | 255 | not null | Current status of the exception defined whether it is treated or pending |
| user | varchar | 255 | null | Identification of the user responsible for treating the exception |
| reference_person_id | bigint | 20 | not null | Foreign key: Reference to reference person’s
|
| entrant_person_id | bigint | 20 | not null | Foreign key: Reference to entrant person’s
|
| transaction_id | bigint | 20 | not null | Foreign key: Reference to transaction’s
|
| reference_person_version | int | 11 | not null | Foreign key: Reference to reference person’s
|
| entrant_person_version | int | 11 | not null | Foreign key: Reference to entrant person’s
|
gbds.status¶
The status table is intended to store the information about nodes and matchers of the GBDS.
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| guid | varchar | 255 | not null | Primary Key; Task ID |
| stimestamp | datetime | 6 | null | Timestamp for status timeout |
| people_count | bigint | 20 | null | Number of people in a node or cluster |
| ul_count | bigint | 20 | null | Number of UL in a node or cluster |
| info | longblob | Up to 4Gb | null | Status information and data |
gbds.apis¶
The apis is used by GBDS to manage multiple APIs active at the same time.
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| api-id | varchar | 255 | not null | ID of the unique API instance |
| hostname | varchar | 255 | not null | Hostname of the node where the API is running |
| port | int | 11 | not null | Port where the API is running |
| type | varchar | 255 | null | Type of the API instance (LEADER, RUNNER, or null) |
On API boot, each API will look for itself in this table. If not found with hostname/IP and port, it inserts itself in the table with a GUID as api-id and type null, meaning not ready for quality extraction. Changes directly applied to this table will be parsed every 15 minutes.
Unsolved Latent (UL) tables¶
The Unsolved Latent tables are used to store information about the UL information and candidates. The tables are described below.
gbds.ul¶
The ul table is designed to store all the GBDS UL’s and is described as follows:
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| uguid | varchar | 255 | not null | Primary key; HBase stored UGUID |
| ul_status | varchar | 255 | not null |
HBase replicates this status |
| creation_time | timestamp | 4 | not null | current_timestamp by default |
| person_pguid | varchar | 255 | null | PGUID of the matched person when
SOLVED |
| person_tguid | varchar | 255 | null | TGUID of the matched person when
SOLVED |
| fragment_id | varchar | 255 | null | Original fragment ID for the fragment that generated the UL |
| fragment_case_id | varchar | 255 | null | Original case ID for the fragment that generated the UL |
| fragment_index | int | 11 | null | Fragment index for the UL. By
default, the index is set as
-1 (index not known, any
index) |
| analysis_user | varchar | 255 | null | User responsible for the UL analysis |
| analysis_timestamp | timestamp | 4 | null | Analysis timestamp |
| group_guid | varchar | 255 | not null | Grouping GUID (for listing linked UL’s) |
gbds.ul_candidate¶
The ul_candidates table is designed to store the candidates of each UL that GBDS keeps
after any REVERSE LATENT MATCH done over enrollments that generated a match against a UL,
and is described as follows:
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| ul_uguid | varchar | 255 | not null | Primary key; UL UGUID |
| person_pguid | varchar | 255 | not null | Primary key; Candidate’s PGUID |
| person_tguid | varchar | 255 | not null | Primary key; Candidate’s TGUID |
| person_index | int | 11 | not null | Primary key; Candidate finger index |
| score | int | 11 | not null | Matching score for the candidate |
| deleted | tinyint | 1 | not null | 0 by default, indicates
whether the candidate was deleted
to be excluded when performing
candidates listing operations |
| minutiae | longblob | Up to 4Gb | null | JSON serialization containing the matched minutiae. It’s a list from the following structure:
|
Notification tables¶
The notification tables are used to store data for auditorship purposes, as e-mails, persons who will be notified, and groups.
gbds.notify_user¶
The notify_user table is designed to store the data of the gbds authenticated user.
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| id | bigint | 20 | not null | Primary Key; User unique ID. |
| username | varchar | 255 | not null | authenticated gbds username |
gbds.notify_group¶
The notify_group table is designed to store the notification groups information.
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| id | bigint | 20 | not null | Primary Key; |
| name | varchar | 255 | not null | Group Name |
| enabled | tinyint | 1 | not null | Defines if the group will be active or not |
gbds.notify_group_email¶
The notify_group_email table is designed to store the emails of a given group.
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| notify_group_id | bigint | 20 | not null | Primary Key; ID of the group. |
| varchar | 255 | not null | Primary Key; Email that belongs to the group |
gbds.notify_user_group¶
The notify_user_group table is designed to store the information of which group
a given user is part of.
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| notify_user_id | bigint | 20 | not null | Primary Key; Reference to user’s
|
| notify_group_id | bigint | 20 | not null | Primary Key; Reference to group’s
|
gbds.people_transparency¶
The people_transparency table is designed to store the information about a given person
and what actions are taken when this person is searched.
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| id | bigint | 20 | not null | Primary Key; |
| pguid | varchar | 255 | not null | Reference to person’s
people.pguid |
| enabled | tinyint | 1 | not null | Indicates if the action is on or off for the given person |
| action | varchar | 255 | null | Action to be taken |
gbds.people_transparency_group¶
The people_transparency_group table is designed to store the information about which groups a
person belongs to.
| Column | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| people_transparency_id | bigint | 20 | not null | Primary Key; Reference to people_transparency’s
|
| notify_group_id | bigint | 20 | not null | Primary Key; Reference to group’s
|
GBDS Settings tables¶
GBDS uses a relational database to store some settings for the API and GBDS. The settings table
is a special table designed to control certain GBDS and GBDS API configurations.
These configurations are stored in the gbds.settings table. The table can contain
configurations present in GBDS API, GBDS, or both following the schema below:
| Type | Type | Size | Value | Additional Information |
|---|---|---|---|---|
| skey | varchar | 255 | not null | Primary key. Parameter key name. |
| stype | varchar | 50 | not null | Local where is the parameter from. API or GBDS |
| svalue | varchar | 4096 | null | Value of the parameter |
| description | varchar | 4096 | null | Description of the parameter |
| stimestamp | datetime | 6 | not null | Timestamp |
| host | varchar | 1024 | null | Hostname of a node |
All settings found inside the table will be written in the respective file, gbdsapi.properties for GBDS API
and application.conf for GBDS, every 15 minutes. Also, it will propagate all updated configurations
on memory in both, API and GBDS.
Note
The parameter value defined at gbds.settings table will propagate to ALL nodes.
This feature is controlled by one configuration parameter in the table, gbds.rdbSystemConfiguration.enabled.
This parameter will enable the overwrite of the values of GBDS and API configuration. Setting the parameter
value to BOTH will make GBDS and API configurations be overwritten.
By default, the following configurations are in the table:
| gbds.settings / File Configuration Key | type |
|---|---|
| gbscluster.min.quality | API |
| gbds.enroll.fingerprints.min-nr-template | API |
| gbds.enroll.face.min-nr-template | API |
| gbds.enroll.iris.min-nr-template | API |
| gbds.enroll.palmprint.min-nr-template | API |
| gbds.enroll.newborn-palmprint.min-nr-template | API |
| gbscluster.enroll.fingerprints.verify.matchthreshold | API |
| gbscluster.update.min.quality | API |
| gbds.transparency.search.identify.request.notify.enabled | API |
| gbds.transparency.search.identify.result.actions.enabled | API |
| gbds.api.logLevel | API |
| gbds.extraction.service | API |
| gbds.extraction.service.face.count | API |
| gbds.extraction.service.ginger.count | API |
| gbds.extraction.service.girl.count | API |
| gbds.extraction.service.hostname | API |
| gbds.extraction.service.initialPort | API |
| gbds.extraction.service.logLevel | API |
| gbds.extraction.service.maxTries | API |
| gbds.extraction.service.linkLibSegfault | API |
| gbds.extraction.quality.service | API |
| gbds.extraction.quality.fillTransactionQualityPropertiesTable | API |
| gbds.faces.extraction.quality.api | API |
| gbds.faces.extraction.quality.background | API |
| gbds.fingerprints.extraction.quality.api | API |
| gbds.fingerprints.extraction.quality.background | API |
| gbds.extraction.quality.service.finger.count | API |
| gbds.extraction.quality.service.face.count | API |
| gbds.extraction.quality.service.initialPort | API |
| gbds.extraction.quality.service.logLevel | API |
| gbds.extraction.quality.service.timeout | API |
| gbds.extraction.quality.service.hostname | API |
| gbds.extraction.quality.service.maxTries | API |
| gbds.extraction.quality.service.linkLibSegfault | API |
| gbds.extraction.quality.service.rows-on-select | API |
| gbds.extraction.quality.service.submitted-queue-factor | API |
| gbds.enroll.face.min.quality | API |
| gbds.update.face.min.quality | API |
| gbds.monitor.url | API |
| gbds.template.face.multiplicity | API |
| gbds.biographicBase.enabled | API |
| gbds.biographicBase.endpoints | API |
| gbds.biographicBase.get.timeout.ms | API |
| gbds.biographicBase.list.timeout.ms | API |
| gbds.biographicBase.logLevel | API |
| gbds.biographicBase.clientID | API |
| gbds.biographicBase.clientSecret | API |
| gbds.biographicBase.lookAllServers | API |
| gbds.log.diagnose | GBDS |
| gbds.ul.boot.scan.enabled | GBDS |
| gbds.boot.scan.ignoreErrorsOnRegion | GBDS |
| gbds.boot.matcher.creation.sleepTime.ms | GBDS |
| gbds.biometric.fingerprint.identify.threshold | GBDS |
| gbds.biometric.fingerprint.exception.threshold | GBDS |
| gbds.biometric.fingerprint.exception.enabled | GBDS |
| gbds.biometric.fingerprint.exception.enroll.min-matches-for-exception | GBDS |
| gbds.biometric.face.identify.threshold | GBDS |
| gbds.biometric.face.exception.threshold | GBDS |
| gbds.peopleList.countFromRDB | GBDS |
| gbds.biometric.face.enabled.threshold | GBDS |
| gbds.driver.logLevel | GBDS |
| gbds.log.loadUnload | GBDS |
| gbds.template.memory.format | GBDS |
| gbds.match.service.enabled | GBDS |
| gbds.match.service.initialPort | GBDS |
| gbds.match.service.logLevel | GBDS |
| gbds.match.service.timeout | GBDS |
| gbds.match.service.templateSend.parallelByModality | GBDS |
| gbds.match.service.linkLibSegfault | GBDS |
| gbds.match.service.maxTries | GBDS |
| gbds.match.service.maxConnectionErrors | GBDS |
| gbds.memory-monitor | GBDS |
| gbds.watchdog.interval | GBDS |
| gbds.watchdog.log.mode | GBDS |
| gbds.watchdog.log.level | GBDS |
| gbds.verifyPostMatch.enabled | GBDS |
| gbds.transparency.search.identify.result.notify.enabled | BOTH |
| gbds.transparency.email-notifier.url | BOTH |
| gbds.transparency.email-notifier.log-level | BOTH |
| gbds.transparency.email-notifier.timeout | BOTH |
| gbscluster.update.consider.fingerprints | BOTH |
| gbscluster.update.consider.faces | BOTH |
| gbscluster.update.faces.verify.matchthreshold | BOTH |
| gbscluster.update.minimum.fingers | BOTH |
Other configurations can be placed on table gbds.settings on rdb and all will be written on API or GBDS file according to configuration type. But the memory reloads on execution time will not be performed for these new configurations, only after API and/or GBDS are restarted.