Shared File Transfer Integration through Mulesoft

File Transfer integration is used to transfer the files from one system to another system using Mule4

Challenges to User/Developer-

Information plays a major role in all the departments. In the Ecommerce Platform, we have tons of information to be shared with customers, vendors, goods, and logistics. Facing issues while sharing all this confidential information with the systems.

Solution: Shared File Transfer integration

Approach-

Create a mule4 flow to connect with other systems using mule connectors (FTP, FTPS, File, and SFTP) to transfer the data

Mule Overview –

The mule runtime engine is a lightweight integration engine that runs Mule applications and supports domains and policies.

Mule Applications –

Mule applications connect systems, services, APIs, and devices using API-led connectivity instead of point-to-point integrations. Mule applications provide functionality for message routing, data mapping, orchestration, reliability, security, and scalability.

Technical Solution –

The file transfer integration application helps us to transfer the information from One System to another System and Vice-Versa. To achieve this we have developed two applications named shared-outbound-file transfer-API and shared-inbound-file transfer-API.

A Shared Outbound file transfer application will be used to transfer the details from our own System to the third-party system.

The flow will be triggered by Mule Scheduler or from the internal team. Once it is triggered it will collect the interface name from the attributes and then we will log the initial request information in our own custom logger named Splunk logger.

Then the flow will read the respective interface file, which contains complete information’s about the transactions like whether it is an inbound or outbound call, to be encrypted or decrypted, inbound or outbound path details, and finally connection details like host, port, username, and password. The flow will throw an error if the file size is Zero. If not the further process will continue like collecting the interface name, source configuration details, target configuration details, encryption details, decryption details, etc.

Shared Outbound File Transfer High-Level Diagram:

Shared Outbound File Transfer High Level Diagram

The initial step is to list all the files from the source system using the source configuration details. The List operation will return the payload in the array format. Using For Each, we are doing iterate and performing the further operations.

Steps followed to achieve the file transfer integration:

Getting the file from the source using the respective read connector(FTP, FTP’s, file, or sftp)
If the pgpEncryption is true we are Encrypting the data which we get from the Source system using pgpEncryptKeyId and pgpEncryptFingerPrint. The AES_256 algorithm is used to encrypt the data.
If the pgpDecryption is true, we are Decrypting the data using the pgpDecryptKeyId, pgpDecryptFingerPrint, and pgpDecryptPassphrase.
If the Compression is true we are Zipping the file in our flow which we collected from the source system.
Writing the file to the target destination.
After the transmission of the source file to the target path, we are archiving the input file to the given path.
Finally, we are deleting the input file in the Source path to avoid duplication.

Based upon the response from the Write (Target) operation we are populating the respective message to the end-user.

The Splunk logger is used to capture the details like input request data, processing data, and target output response data in our flow.

Splunk Logger: It’s a custom Logger to store the log details.

Mule Scheduler: Based upon the cron expression the Mule flow will get triggered. If it is required we can trigger the flow manually’

A Shared Inbound file-transfer: the application will be used to get the details from the third-party system into our own System. The steps are very similar to the shared outbound file transfer application. Only the Operation gets different.

Advantages–

Shared file transfer applications will have all operations in and out. It will be used by other applications to transfer the files. It will reduce our effort and time to transfer all the files.

The calling applications just will have the logger to capture the processing log, HTTP request to call the shared file transfer, and the Set payload.

The re-using concept in the integration world will minimize the man cost, it will increase the efficiency, and the post-production support and bug fix will be very easy.