Tip of the Month for: March 2015

Patience is of "Key Importance" when Running Purges

Our call history indicates that more and more users are cancelling, aborting, stopping, or otherwise interrupting purge programs that are running.  Doing so is almost always going to cause of loss of data and effort on the part of the user and Data Resources both to get the data restored to where nothing of importance is lost.  To try to alleviate the aggravation of cancelling purge programs is causing we want to publicize some key facts about how purges work in general and to assure users that purges are usually running fine, even if they are taking longer than expected or than they did in the past.

• Cancelling out of a purge program with Ctrl+Alt+Delete, Task Manager, shutting down, logging off or by any means, will almost certainly cause you to lose data in the area that you are purging.

• It is extremely rare that a purge program is truly "hung" or stopped.  Even if Windows complains the program is "Not Responding", more than likely the program is still running, but just very busy reading and writing data out to disk and is just too busy to respond to Windows quick enough to where Windows gets "impatient" and shows that message prematurely.  The Not Responding"  message should not be taken literally and should all but be ignored during intensive purge programs. This is a function of how the AcuCobol Runtime and Windows inter-communicate and is unfortunately out of the control of Data Resources and Answers programs.

• If patience is exercised and purge programs are allowed to finish, they will eventually finish on their own and will be successful.

• Certain purges have grown in time over the past few years due to additional features being added which in turn have made these files much larger thus requiring more time to run.  But this should not be considered a problem, and they should be allowed to finish even if requiring substantially more time than in the past.  This includes the P/O Purge, P/O History Purge, Inventory Month End Processing, Inventory Transaction History Purge and the Invoice History Purge.  The reason for the increase in purging times is due to the increase in file sizes to support new features that have been suggested, voted upon and supported by the majority of users.

• If you have access to the server, running Purge programs on the server will run much faster because all of the reading and writing of data does not have to go back and forth over the network.  You will normally see a substantial improvement (decrease in time) between running large purges on the server versus a networked PC regardless of how fast or new the PC is.

• Purge programs start to work by "renaming" the live data files they are about to purge.  They they read thru these renamed temporary work files in their entirety.  For data that is newer than the cut-off date, it is written back out into a new fresh data file.  For data that is older than the cut-off date, it is either written into a history version of the file, or in the case of purging history, skipped entirely which is how older data eventually gets removed from the system entirely.  At the conclusion of a successful purge, the temporary work files are then deleted thus saving that disk space.  The end result is a clean live data file only containing new data.  This process (while it requires more time to run than a program that simply runs thru and deletes out older records) produces a data file that is much more efficient than one with many deleted records.

• But due to how these work, a serious problem is caused when a purge program is aborted in any manner as all of your live data prior to the purge starting is now stored in a temporary work file that you really have no access to.  And the only data that will be accessible is the amount of current data that has been processed up to the point where the purge is aborted.  So the longer the purge runs, the more data you will have.  But still regardless, if a purge is aborted, there will be some amount of data loss.  And not only that, but if not corrected soon, you will be unable to run the same purge again in the future without dealing with the problem.  You may have seen these messages before warning that a "previous instance of this purge did not complete successfully and this can cause a loss of data and you should contact Data Resources" or a similar warning message.

• So the point being is that the purge programs are being used by many users every month and year and if allowed to finish, will finish successfully.  History proves this point.  The amount of time will very greatly depending on the exact purge you are running and your volume of data and the speed of your server, network, and PC.  So just please be patient and let them finish and you will save yourself the grief of losing data and being down while it is recovered.  And again, if you have access to the server, remember they will always run faster on it if that is an option.