7 Advanced features

This section describes several advanced features for some of the developer’s specific needs and optimizations.

7.1 Display frame rate

To show the frame rate in real time, go to the Inspector panel of Pvr_UnitySDK and check the “Show FPS” under the Pvr_UnitySDK Manager component:

_images/7.1.png

Figure 7.1 Opening of FPS

7.2 Open Screen Fade of the Scenes

The SDK provides Open Screen Fade of the Scenes to be turned on through options as follows

_images/7.2.png

Figure 7.2 Opening of Screen Fade

Figure 7.3 can be used to set the Fade Time and Fade Color of the Scenes (time unit is S)

_images/7.3.png

Figure 7.3 Fade Time and Fade Color after Opening the Scenes

7.3 Limited frame rate

The SDK provides the function of limiting frame rate, which is opened by unchecking the Use Default FPS option as follows

_images/7.4.png

Figure 7.4 Opening of limited frame rate

Then enter the desired frame rate value in the FPS text box.

7.4 Configure the size of EyeBuffer

The SDK provides the function of configuring the size of the Eyebuffer, which is opened by unchecking the Use Default RenderTexture option as follows

_images/7.5.png

Figure 7.5 Configuration size of Eyebuffer

Tips：

Developers are advised to use the Use Default RenderTexture to use custom RT sizes only for special needs. For this option developers must understand the following two points：

  • If the RT setting is too small, it will improve performance and reduce latency, but it will also result in lower resolution.

  • If the RT setting is too large, it will reduce performance and increase latency, so it is not recommended that the RT setting exceed the hardware recommended texture size.

7.5 Custom Application Launch Animation

The SDK provides the function of customizing the application to launch animation. Click the menu bar Pvr_UnitySDK/ Splash Screen to open the Splash Screen editing interface, as shown in the figure:

_images/7.6.png

Figure 7.6 Splash Screen option

SplashScreenType contains three types of values: UseUnitySplashScreen, UsePicoSplashScreen, UseDynamicSplashScreen. Each type corresponds to a different application startup animation, and the following describes the configuration methods of the three startup animations.

7.5.1 Launch animation with the Unity app

SplashScreenType selects UseUnitySplashScreen, then click ok, the log message prompts Congratulations when the configuration is complete. At this point, the application startup animation uses the startup animation set in Unity PlayerSetting->Splash Image.

7.5.2 Launch animation with the default app provided by the Pico SDK

SplashScreenType selects UsePicoSplashScreen, then click ok, the log message prompts Congratulations when the configuration is complete. The application startup animation is now using the default startup animation provided by the Pico SDK.

7.5.3 Starting an animation with a custom app

SplashScreenType selects UseDynamicSplashScreen, the Splash Screen interface is as follows:

_images/7.7.png

Figure 7.7 Splash Screen interface

SplashImage: An array of frame animation files that plays when the application starts. The order of the frame animation files in the array should correspond to the order of playback. Image pixel size should be less than 1080x720.

Inside_background: The background image of the application launch animation. Image pixel size should be less than 1080x720.

Use Splash Text: Startup copy when the app starts, check to enable it. After checking, the sub-items appear as shown below:

_images/7.8.png

Figure 7.8 Use Splash Text option

Default Text: The default startup copy, if the system language of the device does not belong to any of the LanguageLocalization, the default startup copy is displayed.

LanguageLocalization: Fill in the corresponding startup copy content for different languages. When the application starts, it will select which startup copy to display according to the system language of the device. If the current system language is not included in LanguageLocalization, the Default Text default startup copy content will be displayed.

FontSize: Starts the copy text size.

FontColor: Starts the copy text color.

TextHeight: The distance from the start of the copy text to the bottom of the screen, in pixels.

Use Carousel: After the marquee, check the marquee function, if the text of the startup copy is too long, it will scroll.

Alignment: Starts the alignment of the copy text.

After all configuration items are configured, click the OK button to save the configuration. The Log message prompts Congratulations to be saved later.

7.6 Eye Tracking

Pico Neo2 Eye supports eye tracking feature. Eye Tracking can track the eye gaze position, optimize rendering performance with Foveated Rendering.You can open Eye Tracking in the Pvr_UnitySDK under Head:

_images/7.21.png

Figure 7.9 Eye Tracking

7.7 Foveated Rendering

Foveated rendering optimizes the ability to render scenes, providing high resolution for the center of the field of view, while peripheral views greatly reduce the details of the scene. The relevant properties are set in the Foveated Rendering option as illustrated by 7.10:

Pico SDK also supports dynamic foveated rendering (i.e. DFR). DFR will apply foveated rendering following movement of eyesight. This feature only applies to Neo2 Eye (for eye-tracking option, refer to 7.6) Foveated rendering options can be found under Head object of Pvr_UnitySDK.

When checking “Foveated Rendering”, a sub option will appear which allows configuring different level of foveated rendering resolution modifying.

_images/7.9.png

Figure 7.10 Foveated Rendering

Foveation Level: Available in Low, Med, and High levels. Each level corresponds to a different Foveation parameter.

_images/7.03.png

ffrGainValue：Ratio of pixel scaling down in X/Y direction, larger value indicates more ratio scaling down.

ffrAreaValue：Define the area in which resolution will not be changed. The circle area using ffrAreaValue as redius and foveated gazing point as the center will keep the original resolution

ffrMinimumValue： Texture coordinate parameter of texture filtering function.

Foveated rendering demonstration：

_images/7.00.png

Level as Low, image is blurred

_images/7.01.png

Level as Med, image is more blurred

_images/7.02.png

Level as High, image is more blurred

Besides using inspector to set value, foveated rendering level and parameters can also be get/set by using interfaces introduced in Chapter 6.8

7.8 VR Compositor Layers

In SDK rendering pipeline, contents in the scene is rendered by LeftEye/RightEye cameras thenrendered to the “Eye Buffer”, after that “Eye Buffer” will be sampled by the ATW thread and finally rendered onto the VR screen.

The Compositor Layers (also known as “transmission layer”) provides a rendering method that is different from “rendering scene content directly into the eye buffer”: this method does not need to render the content to the “Eye Buffer”, but directly “transparently” the content to ATW thread for sampling and synthesis processing, thus avoiding a texture sampling process (rendering the content onto the “eye buffer”), which significantly improves the sharpness of the texture and video.

VR Compositor Layers is more useful when rendering messages, text, videos which are used as “Focus of the Scene”. Meanwhile it can also be used for simple scenes and backgrounds. Currently Compositor Layers supports up to 15 layers (more than 15 layers will not be displayed). And in each scene there can be 1 Equirect Layer and 1 Cylinder Layer at most. Considering drawback to the performance, we recommend limit number of layers to 4 for each scene.

VR Compositor Layer settings are as following:

_images/7.10.png

Figure 7.11 OverLay usage

Detailed descriptions:

  • Overlay Type: set overlay type, default option is Overlay

    • Overlay: Texture will be displayed in front of “Eye Buffer”

    • Underlay: Texture will be displayer behind “Eye Buffer”

      • Note: Underlay relies on alpha channel of render target, when all objects are drawn behind the “Eye Buffer”, developers will need to hollow out an area to display the Underlay texture, A ready shader UnderlayHole.shader (PicoMobileSDKPvr_UnitySDKResourcesMaterials) can be used. Or developers can choose to write their own shaders.

  • Layer Index: set reference value used for composite sequence (lower value means layer will be earlier to be composited)

    • [Camera](Overlay)2/1/0[EyeBuffer]0/1/2(Underlay)

  • Overlay Shape: set overlay shape, default is Quad

    • Quad: Quadrate texture, normally used for Text or Messages in Scene

    • Cylinder: Curved cylinder texture, normally used for curved UI

      • Note 1: Centre of Transform component will be the centre of the cylinder, Transform.scale will be used for size of cylinder, [size.z] is radius of cylinder, [size, y] is height of cylinder, [scale.z] is arc length of cylinder

      • Note 2: When using Cylinder texture, camera needs to be placed inside inscribed sphere of the cylinder. Overlay won’t be displayed if camera is too close to the inscribed sphere surface.

        • Equirect: Sphere texture, normally used for 360/180 video players

  • External Surfaces: Checking this option indicates the layer will obtain texture from Android Surface externally. (e.g. texture of Android Player)

    • Note: When enabled, an Android Surface will be initialized and managed by SDK, meanwhile rendering it directly onto Compositor Layer (We suggest enable External Surface to gain shaper video images, i.e. using Android player plugin to send video texture to Compositor Layer bypassing Unity)

  • Texture: Assign textures for left and right eye

    • Left Texture: Texture to be displayed on left eye.

    • Right Texture: Texture to be displayed on right eye.

7.8.1 Standard 2D Texture

In order to facilitate developers to quickly experience the 2D texture pass-through function, the SDK provides a 2DOverlay and a 2DUnderlay example, developers can directly package and install on the device to experience the effect (Note:Editor can not experience this function).

If the developer applies the feature to their project, follow these steps:

  • Step 1: Open Unity to create an empty scene, find Pvr_UnitySDK.prefab in the

PicoMobileSDK/Pvr_UnitySDK/Prefabs/ folder and drag it into the empty scene, and delete the Main Camera object.

_images/7.11.png

Figure 7.12 Drag into prefab

  • Step 2: Create a Quad in the scene, name it OverlayTransform, then adjust the position of the Quad so that it is in the visible position of the Camera. After determining the display position, uncheck the Mesh Render and Mesh Collider components on the Inspector view panel.

_images/7.12.png

Figure 7.13 Create a Quad

_images/7.13.1.png _images/7.13.png

Figure 7.14 Cancel Mesh components

Tips：The Transmit Layer will use the Quad’s Transform information: position, rotation, scale.

  • Step 3: Bind the Pvr_UnitySDKEyeOverlay.cs script to the Quad created in the previous step and adjust its property parameters on the inspector.

_images/7.14.png

Figure 7.15 Bind script

  • Overlay Type: Set overlay layer type as Overlay

  • Layer Index: Set reference the rendering priority (Smaller value means higher priority for rendering);

  • Overlay Shape: Select the overlay shape as Quad;

  • Textures: Specify the 2D texture to be transparently transmitted (Tips: the left and right eyes specify the same texture, otherwise it will cause the left and right eye content to be different, causing dizziness);

Tips:Developers who need to dynamically modify passthrough 2D textures can call the SetTexture (Texture Texture) interface in the Pvr_UnitySDKEyeOverlay. cs script.

  • Step 4: The last step, package and install to the device to experience the effect.

_images/7.15.png

Figure 7.16 running result

7.8.2 360 panorama texture

Usually we need three steps to achieve “360 panoramic view”:

  1. Create a sphere model in the scene;

  2. Create a “360 Panorama Texture” material and assign it to the sphere model, then modify the shader to remove the front rendering (Cull Front);

  3. Place the Camera in the center of the sphere;

The “transmission layer” function requires only two step:

  1. Create an empty object in the scene and bind the Pvr_UnitySDKEyeOverlay.cs script;

  2. Set the Overlay shape as Equirect type and specify a “360 Panorama Texture”

In order to facilitate the developer to quickly familiarize the 360 panoramic texture transparent transmission function, the SDK provides a 360Overlay example, developers can directly package and install on the device to experience the effect (Note:Editor can not experience this function).

If the developer applies the feature to their project, follow these steps:

  • Step 1: The steps are the same as the Overlay & Underlay examples;

  • Step 2: Create an empty object in the scene, name it 360OverlayTransform, and bind Pvr_UnitySDKEyeOverlay.cs script;

_images/7.16.png

Figure 7.17 Create an empty object

  • Step 3: Set property parameters on the script inspector;

_images/7.17.png

Figure 7.18 Set property parameters

  • Overlay Type: Set overlay layer type as Overlay

  • Layer Index: Set reference the rendering priority (Smaller value means higher priority for rendering)

  • Overlay Shape: Set overlay shape as Equirect;

  • Textures: Specify the same 360 panorama texture for the left and right eyes;

Tips：For Equirect type, we recommend disable Pvr_UnitySDKEye.cs script component of LeftEye and RightEye Camera to reduce rendering resources.

  • Step 4: The last step, package and install on the device to experience the effect

_images/7.18.png

Figure 7.19 running result

7.8.3 Head-Locked & World-Locked Mode

By default, the “transmission layer” is World-Locked mode. If you want to implement Head-Locked mode, you only need to create an empty Transform under the Head node and bind the Pvr_UnitySDKEyeOverlay.cs script.

_images/7.19.png

Figure 7.20 Head-Locked Mode

7.9 Monocular camera

Binocular camera: Provides a separate image for each eye from different angles, and the object will be closer to real life.

Monocular camera: Use a single image to render to both eyes, which effectively reduces rendering.

The way to enable a monocular camera is as follows: Check Use Monoscopic to enable the monocular camera, otherwise use a binocular camera.

_images/7.20.png

Figure 7.21 monocular camera

7.11 Tracking Origin

SDK allows setting the Tracking Origin. The default value is Eye Level. When set to Floor Level, tracking origin will be calculated by the height above the ground detected by headset. (Only applicable on supported devices)

Eye Level is recommended for sitting usage while Floor Level is better for standing usage.

_images/7.24.png

Figure 7.24 Tracking Origin Options

7.12 Single Pass

SinglePass is based on native Unity Single Pass feature. It uses one camera to fulfill stereo rendering, reducing half of Draw Call and Occlusion Culling. Using this feature can increase frame rate dramatically in complex scenes. But SinglePass don’t support post processing.

Check “UseSinglePass” to enable SinglePass mode. Uncheck “UseSinglePass” to disable SinglePass mode.

_images/7.25.png

Figure 7.25 Enable SinglePass