A complete application is a collection of one or more concurrently executing tasks connected by channels. Tasks can be treated as atomic building blocks for parallel systems. They can be joined together, rather like electronic components, by connecting their ports with channels. The connection may be implemented in memory or across an interprocessor link, either directly or via software routing.
Each task is a separate program, either in C with its own main function and its own regions of memory for code and data, or in VHDL; how you code a task depends on the types of processor on which you want it to execute.
You explicitly place each of your tasks on the appropriate processor.
You may have as many tasks on each processor as you wish; you are limited only by the available space. You may even have several copies of a task. Each processor is loaded with only those tasks you have specified; placing a task on one processor does not cause it to be loaded on any other.
A task can include code to deal explicitly with external devices. If you do this, you must place the task on the processor with the devices in question.