7 Advanced features¶
This section describes several advanced features for some of the developer’s specific needs and optimizations.
7.1 Display frame rate¶
To show the frame rate in real time, go to the Inspector panel of Pvr_UnitySDK and check the “Show FPS” under the Pvr_UnitySDK Manager component:
Figure 7.1 Opening of FPS
7.2 Open Screen Fade of the Scenes¶
The SDK provides Open Screen Fade of the Scenes to be turned on through options as follows
Figure 7.2 Opening of Screen Fade
Figure 7.3 can be used to set the Fade Time and Fade Color of the Scenes (time unit is S)
Figure 7.3 Fade Time and Fade Color after Opening the Scenes
7.3 Limited frame rate¶
The SDK provides the function of limiting frame rate, which is opened by unchecking the Use Default FPS option as follows
Figure 7.4 Opening of limited frame rate
Then enter the desired frame rate value in the FPS text box.
7.4 Configure the size of EyeBuffer¶
The SDK provides the function of configuring the size of the Eyebuffer, which is opened by unchecking the Use Default RenderTexture option as follows
Figure 7.5 Configuration size of Eyebuffer
Developers are advised to use the Use Default RenderTexture to use custom RT sizes only for special needs. For this option developers must understand the following two points：
If the RT setting is too small, it will improve performance and reduce latency, but it will also result in lower resolution.
If the RT setting is too large, it will reduce performance and increase latency, so it is not recommended that the RT setting exceed the hardware recommended texture size.
7.5 Custom Application Launch Animation¶
The SDK provides the function of customizing the application to launch animation. Click the menu bar Pvr_UnitySDK/ Splash Screen to open the Splash Screen editing interface, as shown in the figure:
Figure 7.6 Splash Screen option
SplashScreenType contains three types of values: UseUnitySplashScreen, UsePicoSplashScreen, UseDynamicSplashScreen. Each type corresponds to a different application startup animation, and the following describes the configuration methods of the three startup animations.
7.5.1 Launch animation with the Unity app¶
SplashScreenType selects UseUnitySplashScreen, then click ok, the log message prompts Congratulations when the configuration is complete. At this point, the application startup animation uses the startup animation set in Unity PlayerSetting->Splash Image.
7.5.2 Launch animation with the default app provided by the Pico SDK¶
SplashScreenType selects UsePicoSplashScreen, then click ok, the log message prompts Congratulations when the configuration is complete. The application startup animation is now using the default startup animation provided by the Pico SDK.
7.5.3 Starting an animation with a custom app¶
SplashScreenType selects UseDynamicSplashScreen, the Splash Screen interface is as follows:
Figure 7.7 Splash Screen interface
SplashImage: An array of frame animation files that plays when the application starts. The order of the frame animation files in the array should correspond to the order of playback. Image pixel size should be less than 1080x720.
Inside_background: The background image of the application launch animation. Image pixel size should be less than 1080x720.
Use Splash Text: Startup copy when the app starts, check to enable it. After checking, the sub-items appear as shown below:
Figure 7.8 Use Splash Text option
Default Text: The default startup copy, if the system language of the device does not belong to any of the LanguageLocalization, the default startup copy is displayed.
LanguageLocalization: Fill in the corresponding startup copy content for different languages. When the application starts, it will select which startup copy to display according to the system language of the device. If the current system language is not included in LanguageLocalization, the Default Text default startup copy content will be displayed.
FontSize: Starts the copy text size.
FontColor: Starts the copy text color.
TextHeight: The distance from the start of the copy text to the bottom of the screen, in pixels.
Use Carousel: After the marquee, check the marquee function, if the text of the startup copy is too long, it will scroll.
Alignment: Starts the alignment of the copy text.
After all configuration items are configured, click the OK button to save the configuration. The Log message prompts Congratulations to be saved later.
7.6 Eye Tracking¶
Pico Neo2 Eye supports eye tracking feature. Eye Tracking can track the eye gaze position, optimize rendering performance with Foveated Rendering.You can open Eye Tracking in the Pvr_UnitySDK under Head:
Figure 7.9 Eye Tracking
7.7 Foveated Rendering¶
Foveated rendering optimizes the ability to render scenes, providing high resolution for the center of the field of view, while peripheral views greatly reduce the details of the scene. The relevant properties are set in the Foveated Rendering option as illustrated by 7.10:
Pico SDK also supports dynamic foveated rendering (i.e. DFR). DFR will apply foveated rendering following movement of eyesight. This feature only applies to Neo2 Eye (for eye-tracking option, refer to 7.6) Foveated rendering options can be found under Head object of Pvr_UnitySDK.
When checking “Foveated Rendering”, a sub option will appear which allows configuring different level of foveated rendering resolution modifying.
Figure 7.10 Foveated Rendering
Foveation Level: Available in Low, Med, and High levels. Each level corresponds to a different Foveation parameter.
ffrGainValue：Ratio of pixel scaling down in X/Y direction, larger value indicates more ratio scaling down.
ffrAreaValue：Define the area in which resolution will not be changed. The circle area using ffrAreaValue as redius and foveated gazing point as the center will keep the original resolution
ffrMinimumValue： Texture coordinate parameter of texture filtering function.
Foveated rendering demonstration：
Level as Low, image is blurred
Level as Med, image is more blurred
Level as High, image is more blurred
Besides using inspector to set value, foveated rendering level and parameters can also be get/set by using interfaces introduced in Chapter 6.8
7.8 VR Compositor Layers¶
In SDK rendering pipeline, contents in the scene is rendered by LeftEye/RightEye cameras thenrendered to the “Eye Buffer”, after that “Eye Buffer” will be sampled by the ATW thread and finally rendered onto the VR screen.
The Compositor Layers (also known as “transmission layer”) provides a rendering method that is different from “rendering scene content directly into the eye buffer”: this method does not need to render the content to the “Eye Buffer”, but directly “transparently” the content to ATW thread for sampling and synthesis processing, thus avoiding a texture sampling process (rendering the content onto the “eye buffer”), which significantly improves the sharpness of the texture and video.
VR Compositor Layers is more useful when rendering messages, text, videos which are used as “Focus of the Scene”. Meanwhile it can also be used for simple scenes and backgrounds. Currently Compositor Layers supports up to 15 layers (more than 15 layers will not be displayed). And in each scene there can be 1 Equirect Layer and 1 Cylinder Layer at most. Considering drawback to the performance, we recommend limit number of layers to 4 for each scene.
VR Compositor Layer settings are as following:
Figure 7.11 OverLay usage
Overlay Type: set overlay type, default option is Overlay
Overlay: Texture will be displayed in front of “Eye Buffer”
Underlay: Texture will be displayer behind “Eye Buffer”
Note: Underlay relies on alpha channel of render target, when all objects are drawn behind the “Eye Buffer”, developers will need to hollow out an area to display the Underlay texture, A ready shader UnderlayHole.shader (PicoMobileSDKPvr_UnitySDKResourcesMaterials) can be used. Or developers can choose to write their own shaders.
Layer Index: set reference value used for composite sequence (lower value means layer will be earlier to be composited)
Overlay Shape: set overlay shape, default is Quad
Quad: Quadrate texture, normally used for Text or Messages in Scene
Cylinder: Curved cylinder texture, normally used for curved UI
Note 1: Centre of Transform component will be the centre of the cylinder, Transform.scale will be used for size of cylinder, [size.z] is radius of cylinder, [size, y] is height of cylinder, [scale.z] is arc length of cylinder
Note 2: When using Cylinder texture, camera needs to be placed inside inscribed sphere of the cylinder. Overlay won’t be displayed if camera is too close to the inscribed sphere surface.
Equirect: Sphere texture, normally used for 360/180 video players
External Surfaces: Checking this option indicates the layer will obtain texture from Android Surface externally. (e.g. texture of Android Player)
Note: When enabled, an Android Surface will be initialized and managed by SDK, meanwhile rendering it directly onto Compositor Layer (We suggest enable External Surface to gain shaper video images, i.e. using Android player plugin to send video texture to Compositor Layer bypassing Unity)
Texture: Assign textures for left and right eye
Left Texture: Texture to be displayed on left eye.
Right Texture: Texture to be displayed on right eye.
7.8.1 Standard 2D Texture¶
In order to facilitate developers to quickly experience the 2D texture pass-through function, the SDK provides a 2DOverlay and a 2DUnderlay example, developers can directly package and install on the device to experience the effect (Note:Editor can not experience this function).
If the developer applies the feature to their project, follow these steps:
Step 1: Open Unity to create an empty scene, find Pvr_UnitySDK.prefab in the
PicoMobileSDK/Pvr_UnitySDK/Prefabs/ folder and drag it into the empty scene, and delete the Main Camera object.
Figure 7.12 Drag into prefab
Step 2: Create a Quad in the scene, name it OverlayTransform, then adjust the position of the Quad so that it is in the visible position of the Camera. After determining the display position, uncheck the Mesh Render and Mesh Collider components on the Inspector view panel.
Figure 7.13 Create a Quad
Figure 7.14 Cancel Mesh components
Tips：The Transmit Layer will use the Quad’s Transform information: position, rotation, scale.
Step 3: Bind the Pvr_UnitySDKEyeOverlay.cs script to the Quad created in the previous step and adjust its property parameters on the inspector.
Figure 7.15 Bind script
Overlay Type: Set overlay layer type as Overlay
Layer Index: Set reference the rendering priority (Smaller value means higher priority for rendering);
Overlay Shape: Select the overlay shape as Quad;
Textures: Specify the 2D texture to be transparently transmitted (Tips: the left and right eyes specify the same texture, otherwise it will cause the left and right eye content to be different, causing dizziness);
Tips:Developers who need to dynamically modify passthrough 2D textures can call the SetTexture (Texture Texture) interface in the Pvr_UnitySDKEyeOverlay. cs script.
Step 4: The last step, package and install to the device to experience the effect.
Figure 7.16 running result
7.8.2 360 panorama texture¶
Usually we need three steps to achieve “360 panoramic view”:
Create a sphere model in the scene;
Create a “360 Panorama Texture” material and assign it to the sphere model, then modify the shader to remove the front rendering (Cull Front);
Place the Camera in the center of the sphere;
The “transmission layer” function requires only two step:
Create an empty object in the scene and bind the Pvr_UnitySDKEyeOverlay.cs script;
Set the Overlay shape as Equirect type and specify a “360 Panorama Texture”
In order to facilitate the developer to quickly familiarize the 360 panoramic texture transparent transmission function, the SDK provides a 360Overlay example, developers can directly package and install on the device to experience the effect (Note:Editor can not experience this function).
If the developer applies the feature to their project, follow these steps:
Step 1: The steps are the same as the Overlay & Underlay examples;
Step 2: Create an empty object in the scene, name it 360OverlayTransform, and bind Pvr_UnitySDKEyeOverlay.cs script;
Figure 7.17 Create an empty object
Step 3: Set property parameters on the script inspector;
Figure 7.18 Set property parameters
Overlay Type: Set overlay layer type as Overlay
Layer Index: Set reference the rendering priority (Smaller value means higher priority for rendering)
Overlay Shape: Set overlay shape as Equirect;
Textures: Specify the same 360 panorama texture for the left and right eyes;
Tips：For Equirect type, we recommend disable Pvr_UnitySDKEye.cs script component of LeftEye and RightEye Camera to reduce rendering resources.
Step 4: The last step, package and install on the device to experience the effect
Figure 7.19 running result
7.8.3 Head-Locked & World-Locked Mode¶
By default, the “transmission layer” is World-Locked mode. If you want to implement Head-Locked mode, you only need to create an empty Transform under the Head node and bind the Pvr_UnitySDKEyeOverlay.cs script.
Figure 7.20 Head-Locked Mode
7.9 Monocular camera¶
Binocular camera: Provides a separate image for each eye from different angles, and the object will be closer to real life.
Monocular camera: Use a single image to render to both eyes, which effectively reduces rendering.
The way to enable a monocular camera is as follows: Check Use Monoscopic to enable the monocular camera, otherwise use a binocular camera.
Figure 7.21 monocular camera
7.10 Application copyright protection¶
To support copyright protection usage on developer platform and application store, SDK provides application permission validating in development and release phase so developers can protect the copyright.
7.10.1 Validating copyright protection under development¶
In the menu, click Pvr_UnitySDK then Platform Settings to open up Platform Settings menu as shown in the picture below:
Figure 7.22 Platform Settings Option
Entitlement Check Simulation is disabled by default. After checking this option, developers can type in allowed device number and corresponding SN code.Editor mode provides entitlement check validation function, which is used for simulating if copyright is working correctly.
Methods to get SN code of the device:
1.Enter Settings – About – Serial Number
2.Use adb command “adb devices” to get device SN code,
Figure 7.23 Platform Settings Inspector
Related API are listed in Chapter 6.9
7.10.2 Use Copyright Protection when published officially¶
Please refer to chapter 6.9 and find essential APIs for implementing this.
7.11 Tracking Origin¶
SDK allows setting the Tracking Origin. The default value is Eye Level. When set to Floor Level, tracking origin will be calculated by the height above the ground detected by headset. (Only applicable on supported devices)
Eye Level is recommended for sitting usage while Floor Level is better for standing usage.
Figure 7.24 Tracking Origin Options
7.12 Single Pass¶
SinglePass is based on native Unity Single Pass feature. It uses one camera to fulfill stereo rendering, reducing half of Draw Call and Occlusion Culling. Using this feature can increase frame rate dramatically in complex scenes. But SinglePass don’t support post processing.
Check “UseSinglePass” to enable SinglePass mode. Uncheck “UseSinglePass” to disable SinglePass mode.
Figure 7.25 Enable SinglePass