Batch processing and file locations

Image yourself as a data-processing worker. You're sitting peacefully in your cubicle minding your own data. You have a set of work instructions to follow. You have a number of in-boxes and probably an equal number of out-boxes. Together with your colleagues you do your best to get all data processed during working hours.
Now suppose there is another floor in the building where you have a number of your personal clones doing exactly your job, 24 hours a day, 7 days a week. Suppose you could even instruct your clones to pick up a task at a certain time, or repeat it periodically. How cool would that be! You could have them process long-running reports over night or pick up incoming messages 24 hours a day.

How would this work in practice? Well, your clone picks up a task your in-box ... ow ... but wait ... 'your' inbox. But that would be the one on your desk. There's no guarantee any of you clones on the other floor can access it. You surely can't expect them to run down to your desk each time. Your floor might be physically closed during non-office hours. You may lock the drawers of your desk. There are numerous reasons why your clone will not be able to access the data residing on your desk. Bummer.

Hmz ... maybe we could use a sort of centralized in/out-box system where all clones can fetch incoming data and drop the outgoing? Like in 'a wall with labeled boxes on the clones-floor where you can drop the task that need to be executed'. OK, that could work. You could even use the wall yourself if you want to, if your inbox if full for example or when someone wants to drop you a new task while you are away. Convenient!

OK, so let's try this again: your clone has a dedicated in-box in a centralized location. At regular times, it checks if there are any tasks to process. The task-package in the inbox contains both the task description, and the data to process. If there is any external data that is referenced, the reference itself is clearly defined as well. For example: fetch the attached document in in-box 7a for the 'JohnSmith' clones in the central boxes-wall. Looking good!


Now think of the in/out-boxes on your desk as the local storage connected to your PC. This would typically be a file like 'C:\Inbox\Orders\PurchaseOrder_123.edi' or something. Located on your personal computer. A file that is not accessible by anyone else, at least not using the 'C:\Inbox\Orders\PurchaseOrder_123.edi' reference. Even if that reference would exist at someone else's PC, it will not be the your file. So instructing one of your clones to process the data in file 'C:\Inbox\Orders\PurchaseOrder_123.edi' is pointless, since 'C:\Inbox\Orders' references your local storage, which is not accessible to them.

Think of the centralized wall with all the boxes the centralized storage of your company. Like a file server storing files that can be be referenced as '\\fileserver\Inbox\Orders\PurchaseOrder_123.edi' for example.Anyone (with the appropriate privileges, so I'm assuming security is no issue) connected to the company network will be able to access this file. Microsoft named this a UNC path (Universal Naming Convention) where the 'universe' reaches as far as the local network ... but you get the point, right?
So, let me rephrase this: anyone can access this 'universally' defined resource on the network. You can, any of your clones can, your colleagues can.

And the clones? Those are the batch tasks in Ax.
Many processes in Ax have the option to run 'in batch'. This means the processing is not done right away on your machine (your active Ax client), but by one of your clones on the clones floor (AOS). Think of batch processing as dropping a task in the centralized inbox. You don't want to have the result or outcome right away, but you do need the work done. One of your clones will pick it up for you.
And if the task involves a file, you should make it universally available as well. So use a UNC path to reference files with batch processing.

This is where quite some batch-processing frustration comes from: using local file references in combination with batch processing. So, please, keep this in mind when scheduling batch tasks in Ax. It will save you - and your support department - at least the irritation of batch task not handling the files you'd expect them to handle.

AOS and clustering

hi,

There seems to be quite some vagueness when it comes to Ax and AOS clustering.
A few statements to (hopefully) clear things up a bit:

Single AOS

The simplest configuration is an Ax implementation with one single AOS server.

Extra AOS servers can be added in scenarios where you want to increase availability or scale up the Ax environment to handle a larger number of users/connections.
Several possibilities from that point on:

AOS side by side

By default multiple AOS's in a single Ax environment will exist side by side. They do know about each others existence, sync their caches, try not to hamper each other too much,  …
A user connects to a specific AOS as defined in his client configuration. In case there is more than one AOS defined in the configuration the first available will be used.

Adding all available AOS's to one client config file would probably be a smart thing to do. If you only add one node of the cluster to your config file, and this one node is down, you won't be able to connect. If you add all - let's say - 4 nodes, the client config will connect to the first available.

Unless you want to control they way users connect and have a specific department or region connect to a specific AOS of course. I'm thinking latency, bandwidth, ...

Clustered AOS
You can create a load balancing cluster (administration - setup - cluster configuration) and link AOS instances to it. Again two options:

Cluster without load balancer

The naming here is a bit tricky, even if there is no dedicated load balancer set up in a cluster, there will still be load balancing: all AOS's in a cluster implicitly act as load balancers
A user connects to a any of the AOS's as defined in his client configuration. He cannot predict which AOS he will eventually host his session.

The only criteria maintained to balance the load is the number of clients (including business connectors, worker threads, ...) connected to an AOS. If a cluster has two nodes: AOS A and AOS B and A has 3 client connections and B has 4. The next connection will most likely be hosted by A, even if the connecting user's client config only has AOS B in the list of available AOS instances. Most likely, because if AOS A does not respond (in a fashionable time), it will try to connect to B.
In this same example it is possible the load (CPU, memory usage, ...) created by those 3 connections on AOS A is much higher than the load generated by the 4 connections on AOS B. Still the next connection will be hosted by A.

Therefore the client config should have all AOS's listed. You're not sure which AOS will host your session, but you do want all entry point available.

Cluster with load balancer

If you set up an AOS instance to be a load balancer (checkbox 'load balancer' in server configuration) you dedicate  this instance to spread incoming user connection among the non-load balancing AOS's. Ideally the one  with the lowest number of active connections in the cluster will host the first incoming connection. That's all the dedicated load balancer will do for you.

Frankly I don't see the point of setting up a dedicated load balancing server. Maybe when you're dealing with a huge number of concurrent users, logging on and off all the time and you don't want a 'real' AOS to waste time handling those? On the other side: a dedicated load balancer does not consume an AOS license. So it's actually free.

The client config would only contain the dedicated load balancing AOS instance(s), because that's the only valid entry point here. But then again, if your load balancer is down …
Therefore:
- Is it possible to directly connect to a non-load balancer AOS in an AOS cluster with dedicated load balancer? Yes it is.
- Does a non-load balancer AOS in an AOS cluster with a dedicated load balancer also balances load?
Yes it does.

So even if you've got a dedicated load balancer … setting the non-dedicated AOS's instances from the cluster in the client config may be a satisfying plan.

You can go all fancy an set up difference clusters on one Ax environment, for example to split the online and batch processing. Most likely you want client config files per AOS cluster to control who's connecting to which cluster.

Failover

A word on failover might also be in place. An AOS cluster does not mean failover capabilities. It just indicates two or more AOS instances are grouped. That's about it. No automatic moving of sessions from AOS A to AOS B if server A crashes, users will lose their connection to the AOS, manual restart of Ax client by user is required, …
All bad news? No ma'am: an AOS cluster makes sure the load is equally spread over the different nodes. You can add or remove nodes to the cluster online. But in fact … that's about it … and besides the load spreading, a cluster doesn't do more than a well considered config file does.

Other things you might not yet know regarding AOS's:
- Each active not-dedicated loadbalancing AOS instance requires one AOS license. You can install as many AOS instances as you wish, but the number of active instances is limited by the license. You cannot have more active AOS instances than your license allows on the same Ax environment.
- In the online users form you only get to see the SPID's of the users that are on the same AOS as you are
- You can prevent users to connect to an AOS using the 'reject new clients' button on the 'server instances' tab of the online users form. Hit 'accept new clients' to undo this.