GDA-SMDS : Facility EMP
Created by User c0ea1, last modified by Rahul Singh on Dec 19, 2024
Facility EMP
Created by Richa Shukla
Last updated: just a moment ago by Ramesh Varma Kutcherlapati
min read
13 people viewed
Introduction
This document covers the details about the integration of SMDS-EMP platform.
Requirement statement
Maersk is planning to move to event-based integration between producers and consumers. Maersk setup a separate team to manage Event Management Platform (EMP) which will be responsible for environment setup and development activities related to EMP platform. Business will get benefit in terms of no message loss between producer and consumer, can be seamlessly integrated any new consumer or producer application with SMDS, less effort for integration activities, moving to cloud platform etc.
As part of this EPIC, SMDS Facility application will be integrated with EMP platform to produce Facility events. Below are the requirements and the outcome of the first phase.
- Create AVRO schema for Facility interface
- Review the schema with MIM team
- Work with EMP team in terms of schema review, schema registry, topic configuration, connectivity checks etc.
- POC to generate and push JSON message into Kafka topic
- Identify generic events needed for customer core information edits
- Configure required events for KYC
- Work with different consumers for event consume testing if events are required
- Create vessel publish services
- End to End testing
Facility Producer Owner Info:
| Name | Email ID |
|---|---|
| Cindrella Shah | cindrella.shah@maersk.com |
Note : Please connect "noellyn.villar.jasa@maersk.com" for any kind of approval w.r.t connecting/consuming data from Topic # "MSK.geography.facility.topic.internal.any.v1"
Facility EMP ER Diagram :
Facility Consumer Owner Info:
Facility Configuration Details :
| Topic Name | Message Format | Geo Avro Schema | Message Header | Key Columns to get Geo data | Sample Message | Producer Trigger | Pre-Prod Sample Producer Parameter |
|---|---|---|---|---|---|---|---|
| MSK.geography.facility.topic.internal.any.v3 | JSON | OpsFacility_AvroSchema.avscThe Message what ever we are sending and validate through Avro schema.Please find the Above attached Avro schema for Facility.Note: GeoSite will publish to Facility Topic. | While producing message we are appending the below audit details against the message.EventTypeEventDescriptionEventDateTimeMessageId**Sample Data for Header Fields:**EventType : Create/UpdateEventDescription : vesselCreated / vesselDataUpdatedEventDateTime : Wed Feb 13 09:46:36 UTC 2021MessageId : 93f0d94f-c1a9-4d33-8f6c-92cf88f54846 | Facility RowidWhile Producing message to EMP we are passing FacilityRowid to service and fetching data based on that and converting that data into json object | CommercialFacility.jsonOperationalFacility.jsonCustomerFacility.json | SMDS Facility application | kafka.topic: MSK.geography.facility.topic.internal.any.v3kafka.bootstrap-servers:rp-cde-rpx-stage.crb.apmoller.net:9093kafka.username: <Emp Team Provided User Name > kafka.password:<Emp Team Provided password > kafka.schema-registry-url:https://lb-sre-stage-cde.crb.apmoller.net:8081kafka.sasl.jaas.config:org.apache.kafka.common.security.scram.ScramLoginModule required username="<Emp Team Provided User Name>" password="<Emp Team Provided password>";kafka.ssl.truststore-location: <trust store file location path > kafka.ssl.truststore-password: <trust store password > kafka.schema-registry-ssl-protocol:TLSkafka.onprem-sasl-mechanism:SCRAM-SHA-256kafka.ssl.keystore-location:<key store file location path > kafka.ssl.keystore-password:<key store password > key-password:<key store password > kafka.onprem-login-module:org.apache.kafka.common.security.scram.ScramLoginModule**Topic Level Configuration :**ccloud_Retention Period : 72 Hrsccloud_cleanup_policy: compactcompact Mode ON, bcz of that latest message survived in topic based on retention period defined .topic_type: avro replication_flow: g2cLog Compaction is enabled based on the facility id(Geo Id) |