Monday, January 4, 2016

Batch processing and file locations

Image yourself as a data-processing worker. You're sitting peacefully in your cubicle minding your own data. You have a set of work instructions to follow. You have a number of in-boxes and probably an equal number of out-boxes. Together with your colleagues you do your best to get all data processed during working hours.
Now suppose there is another floor in the building where you have a number of your personal clones doing exactly your job, 24 hours a day, 7 days a week. Suppose you could even instruct your clones to pick up a task at a certain time, or repeat it periodically. How cool would that be! You could have them process long-running reports over night or pick up incoming messages 24 hours a day.

How would this work in practice? Well, your clone picks up a task your in-box ... ow ... but wait ... 'your' inbox. But that would be the one on your desk. There's no guarantee any of you clones on the other floor can access it. You surely can't expect them to run down to your desk each time. Your floor might be physically closed during non-office hours. You may lock the drawers of your desk. There are numerous reasons why your clone will not be able to access the data residing on your desk. Bummer.

Hmz ... maybe we could use a sort of centralized in/out-box system where all clones can fetch incoming data and drop the outgoing? Like in 'a wall with labeled boxes on the clones-floor where you can drop the task that need to be executed'. OK, that could work. You could even use the wall yourself if you want to, if your inbox if full for example or when someone wants to drop you a new task while you are away. Convenient!

OK, so let's try this again: your clone has a dedicated in-box in a centralized location. At regular times, it checks if there are any tasks to process. The task-package in the inbox contains both the task description, and the data to process. If there is any external data that is referenced, the reference itself is clearly defined as well. For example: fetch the attached document in in-box 7a for the 'JohnSmith' clones in the central boxes-wall. Looking good!

Now think of the in/out-boxes on your desk as the local storage connected to your PC. This would typically be a file like 'C:\Inbox\Orders\PurchaseOrder_123.edi' or something. Located on your personal computer. A file that is not accessible by anyone else, at least not using the 'C:\Inbox\Orders\PurchaseOrder_123.edi' reference. Even if that reference would exist at someone else's PC, it will not be the your file. So instructing one of your clones to process the data in file 'C:\Inbox\Orders\PurchaseOrder_123.edi' is pointless, since 'C:\Inbox\Orders' references your local storage, which is not accessible to them.

Think of the centralized wall with all the boxes the centralized storage of your company. Like a file server storing files that can be be referenced as '\\fileserver\Inbox\Orders\PurchaseOrder_123.edi' for example.Anyone (with the appropriate privileges, so I'm assuming security is no issue) connected to the company network will be able to access this file. Microsoft named this a UNC path (Universal Naming Convention) where the 'universe' reaches as far as the local network ... but you get the point, right?
So, let me rephrase this: anyone can access this 'universally' defined resource on the network. You can, any of your clones can, your colleagues can.

And the clones? Those are the batch tasks in Ax.
Many processes in Ax have the option to run 'in batch'. This means the processing is not done right away on your machine (your active Ax client), but by one of your clones on the clones floor (AOS). Think of batch processing as dropping a task in the centralized inbox. You don't want to have the result or outcome right away, but you do need the work done. One of your clones will pick it up for you.
And if the task involves a file, you should make it universally available as well. So use a UNC path to reference files with batch processing.

This is where quite some batch-processing frustration comes from: using local file references in combination with batch processing. So, please, keep this in mind when scheduling batch tasks in Ax. It will save you - and your support department - at least the irritation of batch task not handling the files you'd expect them to handle.