Best Practice for handling Large EDI document

There will be a scenario where a large EDI payload with multiple transactions need to be processed by B2B. Processing such a big number (say 10000 transactions - 1 edi payload) need s smart way of handling and passing to backend application.

During this process, the EDI Document processor will generate xml files for all the transaction sets and B2B Engine will process these messages.
By using the following tip.property, we can effectively manage the messages by using the optimised SQL Queries.

oracle.tip.adapter.b2b.edi.optIncomingReq=true

Customizing Control Version in EDIFACT

By default oracle B2B generates the CONTROL message using D3 version. If there is a requirement to customize the version of CONTROL message use the following tip.properties.

oracle.tip.adapter.b2b.edi.FAMessageVersionNumber
oracle.tip.adapter.b2b.edi.FAMessageReleaseNumber
oracle.tip.adapter.b2b.edi.FAAssocAssignedCode

If FAMessageVersionNumber, FAMessageReleaseNumber, FAAssocAssignedCode is specified in tip.properties, we replace those values in generated CONTROL message.

Overdue Acknowledgements - Identify and Take action

How to Handle Overdue Acknowledgments?

Oracle AS B2B provides a feature to configure a timeout value with in which the Functional Acknowledgment has to be received for Outbound business messages.

After successful transmission of EDI Message, B2B must wait for Functional Acknowledgment for configured time. If FA is not received, B2B will enqueue an Error message to B2B Inbound Queue.

set oracle.tip.adapter.b2b.FAWaitTimeOut= time

where time is in minutes.

With FAwaitTimeOut configuration, B2B inserts a Timeout event. If FA is not received with in the time out period, B2B will enqueue an error message.

It is possible to develop a BPEL process to listen on to B2B Inbound queues with B2BERRORUSER and send further notification in the form of fax/phone/SMS etc.

Is batching a Mandatory for Grouping the Transaction set

Even though EDI Batching is the preferred way of grouping the transaction set, it is also possible to handle it by sending multiple transaction set as edifecs XML. B2B accept the multiple transaction set, translate it into EDI document by adding the envelope details to it.

If you find any limitations in this approach , please shoot a mail to ramesh.anantharamaiah@oracle.com

Internal Properties in Oracle B2B

For every EDI document Processing in B2B, there is a need to configure the Envelope information by using Document Protocol parameters in B2B. This information is used by XEngine,

1. To create the envelope for the document sent by Back end application. Xengine create the envelope (Interchange and Group) ,embed the Transaction set and send it to Trading partner.

2. To identify the Incoming Document Type and revision, there by identifying the business action for Agreement Identification.


What is Internal Properties

For every XML schema generated per ecs file while modeling the document in B2B Document editor, it defines Internal Properties. Internal Properties are used to

1. Send the Envelope information to the back end application in case of Inbound flow. By default, XEngine strip off the Envelope information and send only the Transaction set to back end appliction.

2. Override the Document Protocol parameter which is defined in B2B, which is used by B2B to create the envelope.

set oracle.tip.adapter.b2b.edi.FAInternalProperties=true

in tip.properties to avail this feature.

B2B Generates FA/Control messages

Consider the case of B2B generating the Acknowledgment, by default B2B always uses Document protocol parameters to generate the envelope for sending FA/Control. By setting the

oracle.tip.adapter.b2b.edi.FAInternalProperties=true in tip.properties

it is possible to make use of the envelope details of the inbound business document while generating the FA/Control.

Interesting Macros in EDI and HL7

Macros play an important role in XEngine, when there is an use case to fill this data in the B2B Layer, instead of sending it from the Back end application.

1. controlNumber.

In case of Outbound Messages, B2B Generates the control number only in case the Back end application send the EDI message with the Keyword #controlNumber for e.g.

#ControlNumber#

Oracle AS B2B fills in the unique number for Interchange and Transaction set Control Number and uses it for correlating the incoming Control messages.

2. SegmentCount

Segment count for the current EDI Document. Xengine will replace with the approprite values. e.g #SegmentCount#

3. MessageSendingFacility

MSH Position 4.1 e.g #Property(MessageSendingFacility)#

4. MessageSendingApp

MSH Position 3.1 e.g #Property(MessageSendingApp)#

5. MessageReceivingApp

MSH Position 5.1 e.g #Property(MessageReceivingApp)

6. MessageReceivingFacility

MSH Position 6.1 e.g #Property(MessageReceivingFacility)#

Handling Multiple Group ID - Multiple Departments of the same Enterprise

Consider a case where in the Enterprise GlobalCust has Two internal Department with name GlobalChips and GlobalCorp. Trading partner Acme wants to send messages to the two department by referring GlobaclChips and GlobalCorp in the Group ID.

1. Create two Identifier of Type EDI Group ID .
2. One with value GlobalChips and Another with GlobalCorp.
3. Add oracle.tip.adapter.b2b.edi.identifyFromTP = Group in the tip.properties and
restart B2B.
4. Create an Agreement with EDI Group ID as value GlobalChips.
5. Turn off the validation.

Send two message with Application Recipient Code as GlobalChips and GlobalCorp. Both of these messages should be accepted by B2B Engine with the To Party as GlobalCust.

P.S For generating FA similar to Inbound Business message please set

oracle.tip.b2b.edi.FAInternalProperties=true in tip.properties

P.S Make sure to disable the validation.

Another way of enabling this feature is by using the following tip.property.

oracle.tip.adapter.b2b.edi.ignoreValidation=<values>

Provide those identifier which will be different from the configured one.

Few of the sample identifiers are as follows:
InterchangeReceiverID,InterchangeSenderID,GroupReceiverID,GroupSenderID,GroupSenderQual,GroupReceiverQual,InterchangeSenderQual,InterchangeReceiverQual

How To Handle Trimming Spaces in the EDI payload

Ideally there should not be Trimming spaces in the EDI payload , which results in a parsing Error while message processing. This can be handles either by

a. Requesting the customer to Trim the spaces in the end of the payload.
b. Ignoring the Parsing error due to spaces. This can be done by

suppressing the error in $ORACLE_HOME/ip/oem/edifecs/XEngine/config/ErrorSeverityConfig.xml

An error code example is 50420000. In this example, you need to append this towards the end of the ErrorSeverityConfig.xml file.

<ApplyTo>
<Criteria Name="ErrorID" Value="50420000"/>
<SetSeverity SeverityID="0"/>
</ApplyTo>

Large EDI File Processing

This challenge is addressed by writing the payload to a input directory. The corresponding translated payload will also be written to an output directory.

p.s As a pre-requisite always apply the latest patch and set the appropriate heap size as per the performance best practice post.

1. Set the heap size as discussed in the peformance post.
2. Apply the latest patch.
3. You will need to set following two properties in tip.properties

1) oracle.tip.adapter.b2b.edi.incoming.wiremessage.size
.
Specify the size of incoming EDI message that must be treated as large enough
to be kept in file system instead of memory, during message processing. The size
specified is in bytes.

Default is 300 MB (300000000)

2) oracle.tip.adapter.b2b.edi.incoming.wiremessage.directory
Specify directory where large incoming message must be saved.
Example:
oracle.tip.adapter.b2b.edi.incoming.wiremessage.directory=/scratch/mdsharma/edi_files_dir/globalchips

4. For outbound consider File batching as well as Disk caching.

These are the parameters needed to set for this in tip.properties file -

# entries to support large file outbound processing
oracle.tip.adapter.b2b.edi.fileBased=true
oracle.tip.adapter.b2b.edi.outputDirectory=E:\\oracle\\product\\DVAPPSRV01\\ip\\log\\edioutput
#outputSize is expressed in bytes
oracle.tip.adapter.b2b.edi.outputSize=102400

#entries to support no-lock processing of batched outbound messages
oracle.tip.adapter.b2b.sqltablelock=false

#entries to support splitting large batched outbound into smaller batches
#value determines size of smaller batches split off from the large batch
#if this property exists and is set to a value, then #oracle.tip.adapter.b2b.edi.outputDirectory
#and oracle.tip.adapter.b2b.edi.outputSize must be specified.
#if this property exists and is set to a value, oracle.tip.adapter.b2b.edi.fileBased #can be either true or false. true is preferred.
oracle.tip.adapter.b2b.edi.mergeMsgSize=500
oracle.tip.adapter.b2b.qMsgRatio=20

#Entry for file-based processing across multiple B2B instances.
oracle.tip.adapter.b2b.edi.fileBased.directory=E:\\Oracle\\Product\\DVAPPSRV01\\ip\\log\\filebased.

B2BSP - B2B service Provider

B2B Commerce Sdn Bhd is a B2B service provider which we help our customer in daily document exchange process. Our customer can be SME or MNC which have different needs.

What we do daily is receive document from sender via either our client software or standard protocol like AS2 or RNIF and perform data translation or transformation, such as data mapping and transform csv file into EDI file using our in-house product, after all these translation and transformation. Then only send the document to receiver.

What is our value, we help SME to send document to their customer or receive document from their customer. Usually SMEs are not able to do it themselves, due to several problems like budget problem, expertise problem.

We are not only helping SME but also MNC, where they have to exchange document in different format and different protocol such as A want send to B via AS2 but B only receive via RNIF. We help MNC to reach to their level 2 or level 3 suppliers, which usually is not AS2 or RNIF enabled.

Here is a scenario of MNC-A send ORDERS to 1 of its SME supplier, MNC-A send us ORDERS via AS2 in EDI format, we received and transform the document from EDI into XML, then SME supplier logon to e-Supplier to view/print/acknowledge the ORDERS.

Another scenario of A send ORDERS to B send us ORDERS via our client software or AS2 in fixed-length format, we received and transform the document from fixed-length to EDI, then send to B via AS2, so B can import the ORDERS into their backend server, like Oracle EBS or SAP.

There are cases that A will upload the document via our web form and we will perform all the required steps before we send to another party.

Sending 997 in BISAC23020 format

997 BISAC23020 format is similar to X12 3020,except for the version in GS08 segment.Please follow these steps.

1. create the document guidelines for the 850 and 997 from the X12 3020 guildelines.

2. create a group ecs file based on the one from the Xengine directory that has the codelist for GS08 in which replace 003020 with 003020BISAC2

3. create a protocol for version "3020BISAC2". specify the new group ecs from step 2 and install the 2 documents from step 1 into it.

Oracle AS B2B EDI CookBook

EDI Cookbook

ODETTE standard Support in Oracle AS B2B

The ODETTE standard consists of over thirty messages for deployment in the automotive industry and the messages can use either EDIFACT or TRADACOMS service segments. ODETTE messages are supported as part of B2B EDIFACT Document Protocol offering.

NCPDP Support in Oracle AS B2B

B2B supports NCPDP/Script through "EDIFACT over Generic" business protocol because its syntax is based on EDIFACT standard.

Control Number in EDI Document Protocol

Control Numbers are included in all header and trailer segments as an additional means of tracking and confirming EDI transmissions. Control numbers issued in the header segments must match control numbers assigned in the trailer segments. An ISA control number must match the IEA control number, the GS control number must match the GE control number, and the ST control number must match the SE control number in each EDI transmission, or there could possibly be errors. Control numbers can be assigned in the following two ways.

Control Number along with the sender ID provides a unique identifier to identify incoming EDI data from a sender. Following are various control numbers,

Interchange Control Number : Originated by the sender and is typically a sequence number. This together with the sender ID uniquely identifies the Interchange.
Functional Group Control Number :This Uniquely identifies Group.
Transaction Set Control Number : This Uniqely Identifies Transaction Set.

Control Numbers are predominantly used for correlation at various stages; it may be correlation in the back end application or correlation in the B2B application.

Control Number Generated by Back end Application

This is the most likely case as back end application uses the Interchange, Group and Transaction Set Control Number for correlation at various levels and also for correlating the control/Aperack and business messages.

E.g. To correlate 850 EDI message with 997 and 855.


Control Number Generated by B2B

B2B Generates the control number only in case the Back end application send the EDI message with the Keyword #controlNumber for e.g.

#ControlNumber#

Oracle AS B2B fills in the unique number for Interchange and Transaction set Control Number and uses it for correlating the incoming Control messages.

This is not a preferred approach, as most of the correlation has to happen in the back end application.

Various Acknowledgement in EDI

I have already talked a little bit about Functional Acknowledgement. Typically it is called 997 in X12 and is defined in each and every version of X12 such as 4010,4020 etc, where as the same is called Control in EDIFACT and is of version D3.

In the EDI world, people often get confused Functional Acnowledgement with Exchange/Transport level acknowledgement and also the Application level confirmation such as Purchase order Acknowledgment such as 855.

Exchange Level Acknowledgement/Trnasport Level :

Acknowledgement such as MDN i.e Message disposition notification in AS2 or Acknowledgement in EBMS specified the aknowledgement of the business message
in the enterprise only w.r.t meeting the security aspects such as verifying the signed message and decrypting the encrypted message and uncompress it if it is a compressed message and it do nothing w.r.t the payload validity or the business level integration.

Functional Acknowledgement :

Functional Acknowledgement as a Best Practices is handled in most of
the cases using B2B Application level and it caters to validation of the business message w.r.t any defined grammer such as schema,DTD or anything specific to EDI and it also specifies the translation status etc.

It is also possible to handle this in the back end application by delegating the Functioanl Acknowledgement generation to the back end application , however the correllation happens in the B2B.

The flag isFunctionalAck required controls whether an ACK) is needed are not. The flag isFunctionalAclHandledbyB2B controls whether FA is required and it is generated in B2B or it comes from Back End application such as BPEL. This option is useful only when isFunctionalAck required is YES. The flag isFunctionalAclHandledbyB2B should be set to 'YES', unless there is special way ACK is generated outside B2B. Functional Ack is generated using the XEngine in B2B and captures all the translation and validation errors.

To enqueue a query response from BPEL to B2B, populate the replyToMsgID field of
IPMessage with the MsgID field of the incoming message.

Application Acknowledgement :

Application Acknowledgement such as Purchase order Acknowledgement is from the ERP application and caters to the process integration status.

Custom Code List - EDI

Use case : using Interchange Receiver ID qualifier or Interchange Sender ID qualifier ZZ which is not there in the standard code list as part of the Document Protocol Parameter configuration.

Follow the below steps.

1. Modify the interchange ecs file to add the "ZZ" in the standard code list_007.
2. This can be done by editing the Interchange ecs file in the spec builder and adding zz in the S003/007/Standard Code List_007
3. Use the updated ecs while configuring the EDIFACT document protocol parameter as interchange ECS.
4. re-deploy the agreement after validation.
5. Make sure your transactionset ecs do not have any envelope information.

How to test:

Exhange an EDI document whose Interchange Receiver ID as ZZ. It should work :)

Oracle AS B2B -EDI FAQ

1) Is it always mandatory to send the Edifecs XML to B2B for EDI processing?

It is possible to process Native EDI, Edifecs XML and any other flat file using Oracle AS B2B. Following are some of the features with native EDI/Flat File Processing.

· Translation should be switched off with optional validation.
· Oracle AS B2B would not provide features such as Batching, etc. with this configuration.
· It is required to send the complete envelope, as Oracle As B2B will not generate the envelope using Document Protocol Parameters like in case of Edifecs XML processing.


2) What if I have a native EDI file in the back end application can I process in B2B?

Refer to Q1.

3) What if I have a EDI document with multiple transactions, how do I handle in B2B?

Oracle AS B2B Processing is based on Transaction set, hence it is required to split the envelope into multiple transactions and send it to B2B. B2B translates the transaction set into native EDI and generates the envelope using the Document Protocol Parameters.

4) Can I receive EDI document with multiple transaction from a trading partner?

Yes. Oracle AS B2B provides a feature of debatching for inbound messages. Refer to section EDI Debatching for more details.

5) Can I outbound batch a native EDI documents to be sent to B2B from Back end application?
No, Refer Q1.

6) Can I identify the Trading partner using Interchange and group? How to enable this. What is the default method and most preferred method.

Default method and most preferred method is to identify the trading partner using Exchange. For identification based on Interchange/Group please add the following.




Refer to the section Trading Partner Identification for details.



7) Can I customize the group and Interchange ecs files? How?

Yes, it is possible to customize Group and Interchange ecs file using Oracle AS B2B Document editor. Refer to section User Experience while configuring EDI protocol in Oracle AS B2B for details.

8) Is there a need to redeploy the agreement in case of change in the ecs file. Does it require B2B restart?

Refer to section User Experience while configuring EDI protocol in Oracle AS B2B

9) Which database table holds the information about ecs and xsd files?

It is not advisable to access the tables directly as it is driven by the Business Logic layer of Oracle AS B2B.

select a.name from tip_parameter_t a, TIP_DOCUMENTTYPEPARAMETERVAL_T b where a.id = b.DOCUMENTTYPEPARAMETER and a.name like '%ECS%';

Need to link the following 2 tables to get to the DTD/XSD.
· tip_documentdefinition_t
· tip_nativedatatype_T

10) Can I migrate from Gentran system and what is the effort?

It is possible out of the box to migrate a Gentran map to Edifecs ecs file and back. Oracle AS B2B Document Editor provides out of the box features to do this migration. Refer to Document Editor Section for more details.

11) What is the preferred Exchange protocol for EDI?

EDI won’t mandate any Exchange protocol as such. However AS1/AS2 are the most popular exchange protocol which go well with EDI.

12) Can I disable the validation of incoming data against the document protocol parameter? Is it possible to identify the incoming document only based on ecs file?

Yes, it is possible to disable the validation of incoming document against Document protocol parameters.

Oracle.tip.adapter.b2b.edi.ignoreValidation=

Type: String
Default Value: None
Possible Values: EDI Interchange and Group envelope Parameters
Description: Users can specify a list of EDI Envelope Parameters to ignore their validation.


For e.g.: If users enters value InterchangeSenderID, InterchangeSenderQual for this property Oracle AS B2B will not validate Interchange Sender ID and Interchange Sender Qualification for the incoming message.

13) How to change the Recipient and sender Qualifier. Does it require a change in the ecs file?

This needs a change in the ecs file for any deviation from the EDI Standard .

For e.g to change the Interchange Qualifier to ‘zz’ which is not present in the list of default qualifier please do the following steps.

· Edit the Interchange ecs in the B2B Document Editor.
· Add the required qualifier i.e zz in S003-0007 and save the ecs file.
· Update the Interchange ecs file in the Document Protocol Parameter.

14) Can I implement Two-phase commit feature while batching?

Yes. By setting Oracle.tip.adapter.b2b.edi.OneErrorAllError=

Type= String
Default Value : False
Description : If the user wants the b2b server to stop processing all messages in a batch, set this flag to true., otherwise B2B server will process all valid messages and rejects only the messages that fail validation.

15) Is it mandatory to have Functional Acknowledgement in EDI.

No it is optional.

16) Consider the following batch criterion

Canon, 850, 4010, 1
Canon, 810, 4010, 1

Will B2B batch 850 and 810 documents into two separate wire messages?

By specifying No condition using 1 in the criterion column, B2B will create two batches one for 850 documents with revision 4010 and another for 810 documents.

Examples of specifying Criterion

- X12_ONE_TP will combine all X12 messages for one trading partner into one wire message
- X12_ALL_TP will combine all X12 messages for all the partners into one wire message

17) Is it required to turn on validation/translation to generate 997 in B2B.
Yes, it is a must to turn on either validation or translation to generate FA in B2B


18) How to delegate the Functional Acknowledgement generation to Back End Appliaction
Following are the two property of interest while defining the Operational Capability of the Trading partner .

a. Functional Acknowledgement Required : Set it to true to enable the Functional Acknowledgment
b. Is Acknowledgement Handled by Integration B2B : To determine whether B2B handles the Acknowledgement or Back end Application.

Trading Partner Identification in EDI Domain

Oracle AS B2B provides Identification of Trading partner at various levels.

1. Exchange level : This is the default identification based on the exchange Identifier. For e.g if the Exchange protocol used is AS2, then AS2 identifier i.e AS2-FROM is used for identification by comparing against the AS2 Identificer of the Trading partner.

2. Interchange/Group

This is a latest enhancement in B2B where in the identification happens in the order of sequence. Ideally it should be Exchange, Interchange and group. Currently B2B has facility to turn on TPIdentification based on Interchange or Group. This can be done by setting property in tip.properties

Example:
oracle.tip.adapter.b2b.edi.identifyFromTP=Interchange

OR

oracle.tip.adapter.b2b.edi.identifyFromTP=Group

If property is set to Interchange, B2B tries to identify TP using Interchange and if property is set to Group, B2B tries to identify TP using Group.

3. Trading partner Identification Any (newly Added)

If property is set to ANY, B2B will first try to identify TP using Interchange. If no TP found using Interchange, B2B will next try to find TP based on Group.

Example:

oracle.tip.adapter.b2b.edi.identifyFromTP=Any