Information needed before providing EMP/API/DB access
Before giving access to any path of our data consumption, below questions must be answered and documented in confluence :
EMP
EMP is the most preferred way for allowing applications to use Master Data from SMDS.
Mention below to every new consumer in email -
- Our EMP topics are log compacted with all records present on topics for perpetuity. You should start consuming from the latest offset if reload of all records is not desired. If you consume from offset=zero, then all of the CMD records from respective topic will start loading in their DB. This feature can be however used for performance testing in lower environments to present evidence of seamless consumption in case of high outbound.
- As per GDPR compliance guidelines, all Customer & Contact data must be stored within Europe region. Please ensure to design accordingly.
Below details must be documented here - Integration Landscape, before giving access :
- Application Name(s)
- Topics needed (Customer, Contact, Facility, Concern)
- Primary / Secondary Point of Contact
- Team email list (DL)
- Detailed business/benefit case & how you plan to use this data
- Application ID(s) to be added in consumer list
- Tentative dates in SIT, PP and Prod consumption (please send out mail confirmation of successful consumption testing in each env)
- Tentative date for bulk testing in either SIT or PP (so that we don't have any surprises in prod in case of high outbound loads)
API
We shouldn’t expose APIs to external consumers outside SMDS considering higher chances of Major Incidents and tight coupling. APIs should be exposed only to the internal SMDS Portal applications, or some exceptional business benefit cases like Maersk.com Ingestion - customer onboarding.
Our recommended path must be always through EMP consumption, the respective applications can create their own APIs as per needs. We don’t expose experience APIs as master data.
Formal approval is needed from Architect (Rahul Singh), Engineering Managers (Ajit Vijay Daware / User e8ff2 ) before further discussions on API usage.
If we are exposing APIs, below details must be captured and documented here - Integration Landscape
- Application Name(s) & Application Code (for registering source system):
- Primary / Secondary Point of Contact:
- Team Email List:
- Benefit Case & How You Plan to Use This Data:
- Tentative Dates in SIT, PP, and Prod Consumption (Please Send Out Mail Confirmation of Successful Consumption Testing in Each Environment):
- Peak Hour Load:
- Daily Load (Average):
- Business Impact if API is Down:
- Duration for Which API is Needed:
- API Availability Requirements (Uptime and Availability SLAs):
- Security Requirements (Compliance Standards):
- Data Sensitivity (Privacy Regulations):
- Error Handling (Communication to Users):
- Performance Metrics (Expected Response Times Under Peak Load):
- Scalability Requirements (Expected Future Load Increases):
- Integration Points (Other Systems or Applications to Integrate With):
- Testing Requirements (Specific Tools or Processes):
- Documentation (Level Required for API Usage):
- Support and Maintenance (Responsible Parties) :
Database - Read Replica
This is the least preferred route to expose data and must be on most urgent and temporary basis. We can’t expose our database by policy and it also puts the consumers at risk when we make changes for own enhancements.
Formal approval is needed from Architect (Rahul Singh), Data Architects (Gopala Krishnan Rajendran / / Ramesh Varma Kutcherlapati ) Engineering Managers (User e8ff2 / Ajit Vijay Daware ) before further discussions on database usage.
If we are exposing APIs, below details must be captured and documented here - DB Usage
- Why EMP consumption is not feasible
- Application Name(s)
- Primary / Secondary Point of Contact
- Team email list
- Benefit case & how you plan to use this data
- Confirmation that data experts are using this connection since it may impact other production systems
- Start data of usage
- End date of usage