SMDS Integration Access Guide

Architectural guidance for consuming SMDS master data through EMP, API, database read replica, or datalake. This guide describes design considerations, compliance constraints, and requirements that architects should address when integrating with SMDS.

Data Consumption Pathways

Pathway selection depends on latency, volume, query patterns, and operational constraints. EMP is the recommended primary path for change-driven consumption; API for request-response; database read replica only when event streaming is not feasible; datalake for analytical or batch workloads.

Events (RETINA / EMP)

Event-driven consumption via Kafka topics provides near-real-time master data changes. Topics are log-compacted for full retention.

Consumption Strategy

• Production: Start from the latest offset to avoid loading historical backlog and reduce initial sync time.
• Lower environments: Starting from offset=0 can be used for load testing and validating consumption under high outbound volume.

Data Residency (GDPR)

SMDS master data must be stored exclusively within the European region. Design your persistence layer, caches, and any replicated copies to comply with EU-only storage.

EMP Access Requirements

Area	Requirement
Application identity	Application name(s), MAC Id, platform/category (TbM, FbM, MbM, Corporate Platform, BEP)
Consumption scope	Required topics (Customer, Customer Contact, Vessel, etc.)
Ownership	Primary and secondary contacts, team distribution list
Business case	Benefit case and utilization plan
Technical	Application ID(s) for consumer list
Environment timeline	SIT, PP, Production consumption dates; confirm successful testing per environment
Performance	Bulk testing in SIT or PP before production high-load scenarios
Governance	Data retention, error handling, retry strategy
Data scope	Primary fields consumed; whether the application modifies master data and forwards to downstream systems
Design	Architecture and integration design document

API

REST API access for transactional or on-demand reads. Use when you need request-response semantics or cannot consume events.

Design Considerations

• Resilience: Define fallback or degradation when the API is unavailable. Backpressure from SMDS can affect availability.
• Throughput: Document peak-hour and average daily load to align capacity and throttling.
• Compliance: Confirm GDPR and data sensitivity handling for attributes consumed.

API Access Requirements

Area	Requirement
Identity & scope	Requested proxies, application(s), MAC Id, platform/category
Ownership	Primary/secondary contacts, team distribution list
Business case	Benefit case and API usage in the solution
Environment timeline	SIT, PP, Production; confirm successful testing per environment
Load profile	Peak-hour load (req/hr), average daily load (req/day)
Impact	Business impact and major-incident risk if API is down
Resilience	Alternative or backup process when API is unavailable (mandatory)
Duration	How long API access is required
Non-functional	Availability SLAs, security, GDPR affirmation, data sensitivity
Performance	Response time expectations, scalability for future load
Integration	Downstream systems and integration points
Testing	Tools and processes for validation
Documentation	Design and functional documentation level
Support	Support and maintenance responsibilities

Database – Read Replica

Direct database access is a fallback when EMP consumption is not viable. Treat it as temporary; plan migration to event-driven consumption.

When to Use

Use the read replica only when:

• Event streaming does not fit your architecture (e.g., legacy batch, specific query patterns).
• You have a clear migration path to EMP or API.

Operational Impact

Read replica usage affects shared infrastructure. High query volume or complex queries can impact other consumers. Qualified data engineers should own and govern this integration.

Database Access Requirements

Area	Requirement
Rationale	Why EMP consumption is not feasible
Identity	Application(s) accessing the database
Ownership	Primary/secondary contacts, team distribution list
Business case	Benefit case and intended use
Governance	Confirmation that experienced data experts will own the connection
Lifecycle	Planned start and end dates; keep usage time-bound
Access pattern	Tables, fields, and frequency (daily, hourly, ad-hoc)
Performance	Query volume estimate and performance impact assessment
Security & compliance	Encryption, RBAC, GDPR and regional compliance
Migration	Strategy to move from database to EMP when feasible

Datalake

Datalake access supports analytical and batch workloads. Consider sync latency, freshness, and your local storage strategy.

Design Considerations

• Sync latency: Understand impact of data sync delays beyond 24 hours.
• Local storage: If storing master data locally, specify regions (EU, APAC, US) and ensure GDPR alignment for EU data.

Datalake Access Requirements

Area	Requirement
Identity	Application(s), MAC Id, platform/category
Consumption scope	Required topics (Customer, Vendor, etc.)
Ownership	Primary/secondary contacts, team distribution list
Business case	Benefit case and utilization plan
Impact	Impact of downtime or sync delay beyond 24 hours
Storage	Local storage plans and regions
Data scope	Primary fields consumed; modification and downstream forwarding
Design	Usage of this data in your application

Integration Landscape

Platform Approvers

Access requests are reviewed by the SMDS platform team:

Approver	Role	Scope
Rahul Singh	Senior Platform Architect	API, EMP, DB (SMDS-CMD/OPS)
Ramesh Varma Kutcherlapati	Senior Solution Architect	API, EMP, DB (SMDS-OPS)
Mahendra Singh Parihar	Senior Data Engineer	API, EMP, DB (SMDS-OPS)
Gopala Krishnan Rajendran	Senior Data Architect	API, EMP, DB (SMDS-CMD)
Samit Sinha	Data Product Delivery Lead	API, EMP, DB (SMDS-OPS)
Ajit Vijay Daware	Senior Engineering Manager	API, EMP, DB (SMDS-CMD)

Related Resources

• Integration Overview – Integration types and patterns
• API Reference – Endpoints and schemas
• Developer Guides – Technical implementation guidance