Loading...

This chapter explains the (Layer) Picking Mode of the Preview which allows to use content as an interface, i.e. interact directly with it. To give an example, Layer Picking can be used to execute hyperlinks in rendered web pages, so called Browser Assets. For other topics regarding the Preview tab please see the introductory chapter. If you are interested how you can use the Preview's interaction data to draw on a Canvas asset, please see the next chapter "Canvas Editing Mode".

Layer Picking "captures" the position from the mouse pointer and if the mouse is moved in your Master's Preview tab or the Client's fullscreen window the feature recognizes what layer is touched and where exactly on the layer the mouse is located. A layer can now be used as a true interface as mouse clicks can be passed through.
The term "mouse pointer" actually does not only refer to a mouse device but to any device that is recognized by the operating system as a pointing device and can send these input events. This includes for example an AirScan or a touch screen if they are set up to do "clicks". If you are working with an Windows XP system only one input event can be sent, whereas Windows 7 (and above) is able to work with multi-touch events.

The input events can be sent from a local pointing device. Let's say for example you have a Client that renders layers in full screen that should be clicked with an AirScan. Connect the AirScan directly to the Client to receive local input events. Another possibility is to connect the AirScan to another computer which sends the input events to the Client through a so called "Mouse Injection". A Mouse Injection node can be found in the Widget Designer.

If you would like to click in the rendered Preview tab on your Master, simply toggle the Preview to the "Picking Mode".

First example: Layer Picking with Web Page

Please follow these steps to use Layer Picking with a input device connected to a Client that is toggled to fullscreen and shows a web page for example.

In your Pandoras Box project, create a Browser Asset. Right-click in the Project, choose Add Browser and assign an URL in its Inspector. Then assign the Browser to a Layer from the Client. Of course you can work with a Client connected or not connected yet. (Please do not change the default render pass, i.e. leave it assigned to the composition pass.)

To activate click events, open the dialog "Input Event Settings". a) Select the Client in the Device Tree tab and in its Inspector click the according button . b) Toggle the Preview to the Picking Mode, and open the "Show Input Event Settings". This opens a dialog that shows the settings for all systems in the Device Tree.

Activate the Output where you want to click and the option Layers. This passes clicks directly through to the layers and executes underlying links for example. The Widget Designer option is covered in the next example. Now, the mouse capturing works - in the fullscreen window from the Client and - in the Master's Preview if toggled to the Picking Mode in the Output or All Cameras view.

The check box "Show Cursor in Fullscreen" (in the Device Inspector or in the dialog) lets you display the mouse cursor on-top of the Client's fullscreen. In the Master's Preview you will see a special click cursor.

Alternatively you may use "Pointer Layers"; they offer more creative options like displaying a picture or even a Particle System instead of a simple cursor. Depending on the layer's opacity, this Pointer Layer is automatically shown: a) in the Preview tab if you are in the Picking Mode and toggled to the according Output View b) in the Client's small render window c) in the Client's fullscreen render window Optional: Create a Pointer Layer, if you would like to. Right-click the Client in the Device Tree and choose "Add Device" > "Pointer".

Without a Pointer Layer you will see the click cursor in the Preview. With a Pointer Layer you will see that one instead. If you wish to see the click cursor additionally, activate the "Always Show Cursor" button.

Second example: Send Layer Picking data to Widget Designer

In addition to the Preview or Fullscreen interaction, you can also route the data to Widget Designer Device, to execute actions there too. It is also possible to know where the mouse is located e.g. on-top of Layer 1 and work with this data.

Start Widget Designer (4.0 or above) on the same or another PC. In Pandoras Box, create a new project and program layer 1 and 2 next to each other so that your Preview looks something like the example to the left. Please do not change the default render pass, i.e. leave it assigned to the composition pass.

To activate click events, open the dialog "Input Event Settings". a) Select the Client in the Device Tree tab and in its Inspector click the according button . b) Toggle the Preview to the Picking Mode, and open the "Show Input Event Settings". This opens a dialog that shows the settings for all systems in the Device Tree.

Activate the Output where you want to click and the option Widget Designer Devices . This passes the cursor position through a Widget Designer device in the timeline to the Widget Designer application. By the way, the option Layers that we used before can be check or not. The check box "Show Cursor in Fullscreen" (in the Device Inspector or in the dialog) lets you display the mouse cursor on-top of the Client's fullscreen. In the Master's Preview you will see a special click cursor. Alternatively you can use a Pointer Layer as described above.

To add a Widget Designer Device, go to the "Device Types" tab > "Widget Designer" > and drag the "Widget Designer.clib" into the Device Tree. Currently, the device cannot connect and is depicted with a red mark. Select the Widget Designer device in the Device tab to see its Inspector. Enter the IP address from the Widget Designer and the icon will not display the red mark anymore. If it still does, check the IP address and domain number from PB (in the Configuration tab) and from WD as described in the next step.

In Widget Designer, open the Connection menu and choose "PB Configuration". Then make sure that the Domain (1) and IP address (2) under "Master Connection" matches with the PC where the Pandoras Box Master is running on. In Pandoras Box, the IP address is shown in the Assets tab, the Domain in the Configuration tab. Have a look in the section "PB Widget Device Connection" and enable the check box "Enable Connections" (3). You can close the dialog, but remember the button "Input Tester", it opens a dialog that displays incoming information.

Press [Alt + N] and create the following node: Input > Pandoras Box > Layer Interaction (the input nodes "Layer Mouse Input" and "Layer Touch Input" can be used in later projects as well). Make a right-click on the node and choose "Item Properties". In this dialog again, make sure that the IP address matches with Pandoras Box. Leave the dialog open.

So far we have set up the data routing, now we can already use Layer Picking within a fullscreen window of a Client. If you would like to use Layer Picking in the Preview tab, enter the Picking Mode by clicking on the according button on the left site from the Preview and switch the view to the Output you have activated in the Input Event Settings dialog. Please note, that the data is only transferred when the mouse is actually moved.

Now we can move the mouse cursor across the Preview tab or the Client's fullscreen display and see the according data in the Item Properties dialog from the input node in Widget Designer. Alternatively you can open the "PB Network Configuration" dialog again and click the "Input Tester" button. The node reports the Site ID and Device ID as soon as the mouse touches a layer. Of course its opacity value must be greater than zero. X and Y relate to the width and height from the Preview tab or fullscreen window and are measured in pixels.

U and V relate to the layer's texture coordinates. They run from 0 to 1, whilst (0,0) marks the upper left and (1,1) the bottom right corner of a rectangular texture. Of course, having an object e.g. a cone assigned to a layer will result in other coordinates. The UV-coordinates are saved within the object file. The mouse buttons (value 0 or 1) and the wheel (value depending on mouse) report whether the according button was pressed or the wheel was used. The left image shows the reported data in Widget Designer: the mouse touches layer 2 from Server 2 and hovers above its UV-coordinate (0.1,0.5) whilst the XY-coordinate in the Preview tab is (316,160) pixel. If you would like to use the data to click on Widget Designer's Custom Script Buttons, please see its chapter. You can also use further filter nodes, e.g. the "Is in Bounding Box node" to check whether the pointer is in a certain area.

The next chapter explains the Canvas Editing Mode.