Skip to main content

Vendor Architecture Decision Log

👥 Participants: Rahul Singh User bd863 User 729e9 Ajit Vijay Daware Prashila Chandrakant Naik Nanditha Eswar Prasad User 91b26 Praveen Shetty User d3c6d

1. Service Orchestration to Automate Complex Workflows​

Implementation service orchestration to seamlessly integrate and manage interactions between multiple services, automating workflows and managing user tasks. Two options exist have been considered and the proposal is:

  • Camunda BPM: For modelling and executing approval workflows, managing user tasks, and automating processes.
  • Temporal: For managing stateful workflows, handling retries, and ensuring fault tolerance for long-running processes.

Automating service orchestration streamlines complex workflows, reduces manual intervention, and enhances task consistency. This approach also provides robust error handling, transaction management, and improved observability.

Status: PENDING Reason: Temporal is the recommended tool for service orchestration due to its strong support for distributed, long-running workflows and fault-tolerant execution. However, it is currently not designed to handle multi-level BPMN workflows, a feature that Camunda excels in with its native BPMN 2.0 engine.

Currently, SMDS leverages Camunda to manage complex workflows across various domains, utilizing its BPMN modeling capabilities.

As part of the High-Level Design (HLD) phase, we will conduct a deeper evaluation of Temporal’s capabilities, particularly its potential to extend or integrate BPMN-like support. The goal is to assess whether Temporal can be a unified solution for both service orchestration and BPMN workflows, or if a hybrid approach involving both Temporal and Camunda is necessary for optimal workflow management.

2. Master Data Persistence on SMDS-VMDM​

Persistence of master data on the SMDS-VMDM platform, focusing on:

  • Data Modeling: Implementing a comprehensive data model reflecting business entities and relationships accurately. Data Modelling will be aligned with the Maersk Domain Model (MDOM), and may include extensions to the MDOM
  • Data Integrity: Ensuring data integrity is crucial for maintaining consistency and accuracy within the system. While leveraging database constraints and validation rules is essential, it is equally important to consider the role of the relational-object mapping (ORM) framework in managing data integrity. ORM framework (e.g., Hibernate) will be utilised to manage complex relationships and referential integrity at the application level. This approach allows for greater flexibility in handling changes in data relationships without being tightly coupled to the database schema.
  • Data Replication: Implementation of real-time data replication across multiple nodes or geographical locations to maintain up-to-date copies of critical data. This ensures that in the event of a failure in one location, data remains accessible from another, minimizing downtime. Deployment of data across multiple clusters and regions will be implemented to enhance resilience against localized failures or outages. This architecture not only provides redundancy but also enables load balancing, improving overall system performance and responsiveness.

Status: ACCEPTED

3. Data Consumption via API and Events​

Facilitate data consumption through APIs and events, including:

We recommend an API first approach as using events for data distribution, if such data is used to build up own data store at consumer side, is discouraged as it leads to duplicate datasets, potential inconsistencies if not managed correctly and in most cases use of the anti-pattern to build additional APIs in the consumer platform on top of the duplicated data set - APIs that will be replicates of the APIs available, or should be available, in the master data source system.

SMDS will expose both APIs and events, allowing consumers to choose based on their usage patterns, functional, and non-functional requirements. The decision on which to use should be collaboratively aligned between SMDS and the consuming application to ensure optimal integration, as per MIP guideline.

  • API Design: Creating RESTful APIs with versioning, pagination, and rate limiting to ensure scalability and reliability.

APIGEE using Stargate will be used for API gateway. The services will be protected using Forgerock issued access token. Coarse grained access control will happen on API gateway and fine grained access control will happen on the microservice cluster using methodologies as described in section 9

  • Event Integration: Configuring event producers and consumers for real-time data streaming. Implementing dedicated events for:
    • Notification Event: An event will be generated for email communication whenever a vendor creation, vendor update, or a workflow state change occurs. This event will be published to an event bus (e.g., Kafka), allowing subscriber to consume it and trigger specific email notifications. Subscriber will use this event to automatically send tailored email notifications to procurement users, data stewards, and external stakeholders. In addition to conveying the vendor creation/update details or workflow state changes, the event payload will include:
      • Validation results: Upon receiving a payload into SMDS, the system performs a series of validation checks, including field validation, and business logic validation. If any validation errors are encountered (e.g., missing mandatory fields, incorrect data formats, or business rule violations), the event will contain a detailed error report. This error report can be consumed by subscribers to generate notifications with specific error messages, allowing stakeholders to quickly identify and rectify issues.
      • Duplicate vendor warnings: Providing real-time notification about potential duplicate vendor entries, enabling timely corrective action.
    • Workflow Event: When a vendor create/update request is submitted and enters the workflow, or transitions between different workflow stages (e.g., approval, rejection), a dedicated workflow status event is triggered. This event will notify subscribers about the current state of the request, providing real-time updates as it progresses through various workflow stages
    • Vendor Master Data Event - Once vendor data is created, updated, or initiated successfully in the system, a Vendor Master Data Event is triggered. This event is responsible for publishing the updated vendor information to all subscribers in real-time, ensuring data consistency and synchronization across integrated systems.

Status: ACCEPTED

4. Implementation of a Rule Engine Service​

Implementation of a rule engine service for validating incoming data, leveraging wrapper services, and utilizing a service orchestration pattern to enable the use of multiple backend services and external providers for specific validations. The solution will focus on flexibility, allowing dynamic selection and execution of validation services based on the type of data and business logic. Additionally, it will ensure scalability and maintainability by decoupling validation logic from individual services, facilitating easier integration of new providers or validation rules as requirements evolve. The architecture will prioritize high availability, fault tolerance, and optimized performance to handle large volumes of data efficiently.:

  • Rule Definition: Creating and managing validation rules for generic country-specific rulesets, including address, phone, email, bank, and tax information. These rules will ensure robust data validation and enforce quality checks to maintain data integrity and compliance with regional requirements. The rule engine will be responsible for dynamically applying the appropriate validation logic based on the type of data, ensuring consistency across multiple data sources and guaranteeing that all inputs meet the required standards before processing.
  • Integration: Integrating the rule engine with existing data pipelines to enforce validation rules before data ingestion.
  • Error Handling: Implementing mechanisms for error reporting and corrective actions when validation fails.

The rule engine will be natively developed and integrated within the SMDS system, allowing for efficient management of business rules through service orchestration. This approach eliminates the need for external rule engine tools, ensuring full control over the logic and functionality within the system's architecture.

Status: ACCEPTED

Specialized Validation Services Include:

  • Generic Country Rules Validation – The rule engine will be capable of validating the required fields for vendor master data, ensuring that country-specific regulations and mandatory fields are met. It will dynamically apply validation rules based on the country, ensuring consistency and compliance with local requirements for fields such as tax ID, bank details, address, and other critical vendor information.
  • Address Validation: Use geocoding and address formatting services to standardize and verify addresses. Address validation product selection pending.
  • Phone/Email Validation: Implement regular expression checks and verification services to ensure valid contact information. Informatica capability available for phone number and email validation.
  • Bank/Tax Validation: Integrate with external verification services to confirm bank account details and tax identification numbers. Tax validation currently supports EU, AU and NZ. Bank validation product selection is pending .

5. Unified Portal for Internal and External Users​

Develop a unified portal for internal and external users, incorporating:

  • Role-Based Access Control (RBAC): Implementing fine-grained access controls based on user roles to ensure data security and compliance.
  • Integrated Workflows: Designing workflows for tasks like vendor onboarding and procurement approvals to streamline processes.
  • Real-Time Data Access: Providing up-to-date information on procurement, vendor data, and status updates.
  • Self-Service Capabilities: Enabling suppliers to update information and track interactions through a user-friendly interface.
  • Collaboration Tools: Integrating messaging, notifications, and shared workspaces to enhance communication.

The portal will improve collaboration, reduce manual processes, and offer a centralized access point for all stakeholders, leading to greater efficiency and better user experience.

Status: ACCEPTED Reason:

  1. Ownership of various vendor portal functionalities is yet to be finalized, with responsibilities for specific features still under discussion and finalisation
  2. Micro-frontend Considerations – As mentioned by Sourav, the CIAM team will develop vendor onboarding features on https://accounts.maersk.com using a micro-frontend architecture for improved reusability and modularity. However, the feasibility of implementing specific use cases, such as BBU and vendor master data workflows, with micro-frontends is yet to be evaluated. Further analysis is needed to assess compatibility, performance, and integration with existing systems.

6. Reference Data Management​

Accurate management of reference data (tax codes, company codes, purchasing organizations, spend types, bank details) is critical for effective vendor master data management and operational efficiency. It ensures that data used across systems is consistent and up-to-date.

  • Initial Load: Functional/business owners will provide reference data for initial ingestion into the SMDS application via backend processes.
  • Ongoing Management: Reference data will be maintained through a management portal on the vendor portal, allowing for updates and modifications.

🔍 Rationale: The option to automate the management of reference for all objects across all systems would be unfeasible in the time line of the programme and therefore a manual approach has been recommended. This has been agreed with the FPO.

Status: ACCEPTED Reason:

  • The process for managing reference data ingestion is still in the finalization phase, requiring further discussions to outline clear procedures.
  • Development of the reference data UI management portal is currently underway, aimed at facilitating user-friendly access and updates to the data.
  • Collaboration between the Technology and Product Owner teams is essential to establish a refined and comprehensive list of reference data items.
  • It is important to highlight that certain reference data may not be manageable through the portal due to inherent technical challenges, necessitating a careful assessment of data handling capabilities.

The steps for managing reference data through the portal will be initiated during Phase 1, laying the groundwork for streamlined data governance and accessibility. This phase will focus on establishing clear workflows, user roles, and data validation processes to ensure effective management and integrity of reference data.

7. Document Management​

Status : ACCEPTED

Document Management solution offered by TSE will be used to store necessary documents related to vendor onboarding.

Document Management which is offered as a service by TSE uses OpenText xECM (a licensed product) and Azure BLOB for various document management storage.

Below are some of the capabilities the OpenText xECM solution provides:

  • Storing and linking document metadata
  • Retention and disposal of documents
  • Ability to make API calls to automate creation of documents etc
  • Extensive Workflow capabilities
  • Very powerful Search Engines
  • Document versioning
  • Security through RBAC from Open Text Directory Service (OTDS)
  • Ensuring regulatory compliance and storing the documents into correct Azure region based on country of document collection

8. Identity​

Status: ACCEPTED

The login journey for both internal and external users will be owned by cIAM team. Forgerock will be used to store external user credential and their permission. Azure AD will be used for internal user credential and their permission. This eases the joiner, leaver process for internal users.

It is the responsibility of the company admin to review the permission of their user from the settings page offered by cIAM team and revoke access if their employee leaves the organization.

9. Access Management​

Status: ACCEPTED

Role Based Access Control (RBAC), Relationship Based Access Control (ReBAC) and Attribute Based Access Control (ABAC) will be used for Access Management.

Forgerock, Open Policy Agent and Anchor will be used for RBAC, ReBAC and ABAC. This will offload access management logic from microservice layer onto Open Policy Agent which will act as Policy Enforcement Point (PEP) and Policy Decision Point (PDP).

The high level solution is documented here

10. Deduplication Mechanism​

Status: ACCEPTED

For deduplication, we will leverage Elasticsearch to efficiently identify and match duplicate records. By utilizing its powerful search and indexing capabilities, Elasticsearch will enable us to implement sophisticated algorithms for detecting similarities across datasets, ensuring data integrity and accuracy throughout the system. This approach will enhance our ability to maintain a clean and reliable vendor master data repository.

The attributes to be considered for the Match & Merge process have been defined in the Business Requirements Document (BRD). It is essential to revisit these attributes and refine them during the Phase 1 , to ensure alignment with project objectives and data management standards. This review will facilitate a comprehensive understanding of the criteria necessary for effective deduplication and merging of records.

11. DQ Tools​

Status : ACCEPTED

Power BI dashboard will be utilized to showcase Data Quality (DQ) metrics, similar to the existing Customer metrics dashboard.

Key Actions:​

  1. Attribute Identification: Compile a comprehensive list of attributes that need to be considered for building the dashboard as part of the user stories definition. This will ensure that all relevant data points are captured and represented effectively.
  2. Role-Based Access Control (RBAC): Implement RBAC-based controls for dashboard navigation to ensure secure access and functionality tailored to user roles.

These decisions will be finalized before the start of Phase 3

Was this page helpful?