COMPANY DATA ARCHIVE

Company Data Archive Best Practices

BY PROFESSIONAL ADVANTAGE - 27 July 2020 - 2 MINS READ

So, you’re about to run your first Company Data Archive (CDA)? Depending how many years of data your company needs to archive we know this process can take a few hours or a few days. We’ve created this ‘Step by Step Guide’ to help your archiving process run smoothly.

1. Running Your First Archive

Scenario

If this is your first-time using Company Data Archive (CDA),you likely have many years of data you’ll need to archive. However, attempting to archive all seven years of data, across all modules, all at one time, will very likely take days to accomplish. For example, you may have 10 years of data in your live company, but you determine it is only necessary to keep the most recent three years in your live company. It is our recommendation to split this initial archive into several pieces.

  1. First, archive just Master Data.
  2. Then, archive GL for just your oldest fiscal year.
  3. Next, archive GL one year at a time up until the most recent year you want to archive.
  4. Follow this process for all other modules.

Utilize the preview window to help gauge how long it takes to archive different amounts of data. For some modules, you will be able to archive a year or more at one time. Other modules will be larger, and you may only want to archive six months at a time. For modules with less activity, you will be able to archive more than one module at a time.

This initial “catch up” archive may take you several evenings or weekends to accomplish. After that, archiving does not need to be done every day. Based on your initial archive experience, you will develop a feel for how often you need to archive the various modules.

Recommended Action

  • If you have large amounts of data to archive, we recommend you select smaller periods of time. Run the archive during off peak hours when there are no other processes/jobs running (utilizing all available resources). Archiving one module at a time will help improve performance. The archive may run faster for some modules compared to others, depending on the amount of data CDA has toprocess.
  • Run the “Transfer Master Data” one time before beginning the archive of your modules. The Transfer Master Data is moving only “non-transaction” information.(For example, customers, addresses, and accounts).
  • It helps to have three times as much disk space as the amount of data you plan to archive at a time.
  1. Run an Archive with only RM selected (just a ‘transfer’ is fine)
  2. Record how many customers are archived for any given period of time (ex. 10 minutes).
  3. Stop the archive.
  4. Perform the same operation with SOP.
  5. Use that number to calculate a rough time estimate.
  6. If you have limited time for the archive to run, adjust your cutoff date to reduce the number of customers that will be archived.

Important To Note

The number of customers/time you calculated is a very rough estimate. Your estimate of archive time will be significantly lower than the actual time it will take to archive. We recommend adding an additional 10-15% buffer to your time estimate.

For example, you have 1,000 customers that will be archived for the date range given. While the first 100 may take 10 seconds, the last 500 could have 10 times as many transactions as the first 100.

Since you can pause and resume the archive, it is unimportant to finish the archive for a cutoff date in one session. When you pause, CDA finishes archiving the current customer, or vendor, and stops. You can repeat these steps until your archive is finished.

For example, you could start an archive Friday afternoon, pause it on Monday morning, and resume it the following Friday – it will pick up where it left off. When you pause, it finishes archiving the current customer or vendor and then stops. You can repeat these steps until your archive is finished.

2. Document Did Not Archive

Scenario

To inquire on a document that did not archive by using the “Will Document Archive?” window in your live company. This option can be used in the ‘Sales and Purchasing’ series.

Example: ‘Live Company’ Inquiry

  1. SelectInquiry > Purchasing > Transactions by Document
  2. Searchby document date
  3. Selectthe ‘From:  To:’ option
  4. Enteryour archive date in the To: field
  5. Select‘Redisplay’
  6. Locatethe document that didn’t archive
  7. Selectthe document
  8. Select‘Will Document Archive’ from the Additional menu
  9. Re-enteryour cutoff date and then tab off

The apply chain will appear with a report containing the document restricting your desired document from archiving along with a description of why it did not archive.

The window will look similar to this:

 

Archive Criteria

The first SQL table CDA reviews when the archive process begins varies for each module. The following table shows each module with it’s corresponding ‘Driving Table’

Module Driving Table Date Used
GL GL30000 HSTYEAR
BOM BM30100 PSTGDATE
IV IV30300 DOCDATE
IVD IVC30101 DOCDATE
PM PM30200 DOCDATE
POP PO POP30100 DOCDATE
POP RCT POP30300 receiptdate
RM RM30101 DOCDATE
SOP SOP30200 DOCDATE
UPR UPR30100 CHEKDATE

Archive Criteria (continued)

This next table shows the criteria considered when archiving for each module. Each module is specifically designed to transfer information based on how users would expect this information to flow. If data does not match this criterion, then it will not be removed.

Module Name Transfer / Remove    
General Ledger Document Date  
Bank Reconciliation Document Date > > Must be Reconciled
Payables Management Payment (Credit Documents) > > Document Date
Payables Management Invoices (Debit Document) > > Must be Applied to a Payment Located on or before the Archive date.
Purchase Order Purchase Orders > > Document Date
Purchase Order Receiving > > Receipt Date If invoiced, it must be paid to be archived.
Receivables Management Payment (Credit Documents) > > Document Date
Receivables Management Invoices (Debit Document) > > Must be Applied to a Payment Located on or before the Archive date. If a debit document is voided, the document date is used
Inventory Document Date > > Must be Fully Applied
Sales Order Processing Document Date > > Must be Fully Applied
Inventory Document Date  
Bill of Materials Posting Date  
US Payroll Check Date  

If you wish to transfer and/or remove GL information, you must choose a date that falls within a historical year. You are able to transfer only completed historical GL years.

Archive Criteria (continued)

Each module is specifically designed to transfer information under the same manner that users would expect information to flow. If data does not match this criterion, then it will not be removed.

RM and PM module

  • The entire apply chain must be within the archive date, meaning the invoice and all the credit documents applied to it.
  • This also includes all of the other debit documents the payments are applied to, etc. that are before the archive date.

Note: If any document is after the archive date, the entire chain stays and will not be archived.

SOP History module

  • The RM document that is created from the SOP document posting must be in RM History, not RM OPEN, or the SOP History document will not archive.
  • When SOP documents are posted to the SOP 30XXX tables, they also create RM20101 (RMOPEN) documents. The SOP 30xxx records are not moved if the RM20101 rows they created are still in the RM Open table and have a current trx amount (not fullypaid off).

POP History module

  • the PM document that is created for the POP document posting must be in PM History, not in PM OPEN, or the POP History document will not archive.

The PM criterion includes the following:

Payment (Credit Documents) >> Document Date Invoices (Debit Document)>> Must be applied to a payment which is on or before the Archive date.

If a debit document is voided, the document date is used.

3. Inventory and Payables Management:

Scenario

RM and PM work by choosing all documents before the cutoff/archive date and placing them in a temp table. They remove any rows where the “entire apply chain” is prior to the archive date.

Payables Management (PM)

Rows from the PM30200 table that are Archive Dates are inserted into a temp table.

Transactions that are not complete* as of the archive date are removed from the temp table.

*not complete meaning a record in the apply chain has a document date after the archive date.

Transactions that are already in the archive company'sPM30200 table (DOCTYPE,VCHRNMBR) are removed from the archive table.

The following tables are transferred:

  • 'PM30200'joined to the temp table on VCHRNMBR and DOCTYPE
  • 'PM30300'joined to the temp table on VCHRNMBR and DOCTYPE
  • 'PM30600'joined to the temp table on VCHRNMBR and CNTRLTYP
  • 'PM30700'joined to the temp table on VCHRNMBR and DOCTYPE
  • 'PM00400' joined to the temp table on VCHRNMBR and CNTRLTYP
  • 'MC020103'joined to the temp table on VCHRNMBR and DOCTYPE

Inventory (IV)

Rows/transactions from the IV30300 table that are <=Archive Date are inserted into a temp table.

Transactions already in the archive company's IV30300 table(DOCNUMBR, DOCTYPE, LNSEQNBR) are removed from the archive table.

The following tables are transferred:

  • 'IV30200'joined to the temp table on DOCNUMBR and IVDOCTYP and TRXSORCE
  • 'IV30300'joined to the temp table on DOCNUMBR and IVDOCTYP and TRXSORCE
  • 'IV30301'joined to the temp table on DOCNUMBR and IVDOCTYP and LNSEQNBR
  • 'IV30400'joined to the temp table on DOCNUMBR and IVDOCTYP and TRXSORCE and LNSEQNBR
  • 'IV30500'joined to the temp table on DOCNUMBR and IVDOCTYP
  • 'IV30600'joined to the temp table on DOCNUMBR and IVDOCTYP and LNSEQNBR
  • 'IV30100'joined to the temp table on TRXSORCE
  • 'IV30302'joined to the temp table on DOCNUMBR and IVDOCTYP and LNSEQNBR

4. Upgrades and Support:

The latest release of Company Data Archive can be found on the Professional Advantage website.

https://www.profad.com/forms/cda-download/

  1. Under the Solutions menu, select ‘Company Data Archive’
  2. Navigate to the content block: ‘Partners & Customers’
  3. Select ‘Download Software’
  4. Complete the form to proceed to the download page
  5. To request an earlier version of Company Data Archive, please email profad@profad.com

Write a Comment