1. Introduction
The Web Neural Network API defines a web-friendly hardware-agnostic abstraction layer that makes use of Machine Learning capabilities of operating systems and underlying hardware platforms without being tied to platform-specific capabilities. The abstraction layer addresses the requirements of key Machine Learning JavaScript frameworks and also allows web developers familiar with the ML domain to write custom code without the help of libraries.
For an illustrated introduction, please see the explainer.
2. Use cases
2.1. Application Use Cases
This section illustrates application-level use cases for neural network inference hardware acceleration. All applications in those use cases can be built on top of pre-trained deep neural network (DNN) [models].
Note: Please be aware that some of the use cases described here, are by their very nature, privacy-invasive. Developers who are planning to use the API for such use cases should ensure that the API is being used to benefit users, for purposes that users understand, and approve. They should apply the Ethical Principles for Web Machine Learning [webmachinelearning-ethics] and implement appropriate privacy risk mitigations such as transparency, data minimisation, and users controls.
2.1.1. Person Detection
A user opens a web-based video conferencing application, but she temporarily leaves from her room. The application is watching whether she is in front of her PC by using object detection (for example, using object detection approaches such as [SSD] or [YOLO] that use a single DNN) to detect regions in a camera input frame that include persons.
When she comes back, the application automatically detects her and notifies other online users that she is active now.
2.1.2. Semantic Segmentation
A user joins a teleconference via a web-based video conferencing application at her desk since no meeting room in her office is available. During the teleconference, she does not wish that her room and people in the background are visible. To protect the privacy of the other people and the surroundings, the application runs a machine learning model such as [DeepLabv3+], [MaskR-CNN] or [SegAny] to semantically split an image into segments and replaces segments that represent other people and background with another picture.
2.1.3. Skeleton Detection
A web-based video conferencing application tracks a pose of user’s skeleton by running a machine learning model, which allows for real-time human pose estimation, such as [PoseNet] to recognize her gesture and body language. When she raises her hand, her microphone is automatically unmuted and she can start speaking on the teleconference.
2.1.4. Face Recognition
There are multiple people in the conference room and they join an online meeting using a web-based video conferencing application. The application detects faces of participants by using object detection (for example, using object detection approaches such as [SSD]) and checks whether each face was present at the previous meeting or not by running a machine learning model such as [FaceNet], which verifies whether two faces would be identical or not.
2.1.5. Facial Landmark Detection
A user wants to find new glasses that beautifully fits her on an online glasses store. The online store offers web-based try-on simulator that runs a machine learning model such as Face Alignment Network [FAN] to detect facial landmarks like eyes, nose, mouth, etc. When she chooses a pair of glasses, the simulator properly renders the selected glasses on the detected position of eyes on her facial image.
2.1.6. Style Transfer
A user is looking for cosmetics on an online store and wondering which color may fit her face. The online store shows sample facial makeup images of cosmetics, and offers makeup simulator that runs a machine learning model like [ContextualLoss] or [PairedCycleGAN] to transfer the makeup style of the sample makeup image to her facial image. She can check how the selected makeup looks like on her face by the simulator.
2.1.7. Super Resolution
A web-based video conferencing is receiving a video stream from its peer, but the resolution of the video becomes lower due to network congestion. To prevent degradation of the perceived video quality, the application runs a machine learning model for super-resolution such as [SRGAN] to generate higher-resolution video frames.
2.1.8. Image Captioning
For better accessibility, a web-based presentation application provides automatic image captioning by running a machine learning model such as [im2txt] which predicts explanatory words of the presentation slides.
2.1.9. Text-to-image
Images are a core part of modern web experiences. An ability to generate images based on text input in a privacy-preserving manner enables visual personalization and adaptation of web applications and content. For example, a web application can use as an input a natural language description on the web page or a description provided by the user within a text prompt to produce an image matching the text description. This text-to-image use case enabled by latent diffusion model architecture [LDM] forms the basis for additional text-to-image use cases. For example, inpainting where a portion of an existing image on the web page is selectively modified using the newly generated content, or the converse, outpainting, where an original image is extended beyond its original dimensions filling the empty space with generated content.
2.1.10. Machine Translation
Multiple people from various countries are talking via a web-based real-time text chat application. The application translates their conversation by using a machine learning model such as [GNMT] or [OpenNMT], which translates every text into different language.
2.1.11. Emotion Analysis
A user is talking to her friend via a web-based real-time text chat application, and she is wondering how the friend feels because she cannot see the friend’s face. The application analyses the friend’s emotion by using a machine learning model such as [DeepMoji], which infers emotion from input texts, and displays an emoji that represents the estimated emotion.
2.1.12. Video Summarization
A web-based video conferencing application records received video streams, and it needs to reduce recorded video data to be stored. The application generates the short version of the recorded video by using a machine learning model for video summarization such as [Video-Summarization-with-LSTM].
2.1.13. Noise Suppression
A web-based video conferencing application records received audio streams, but usually the background noise is everywhere. The application leverages real-time noise suppression using Recurrent Neural Network such as [RNNoise] for suppressing background dynamic noise like baby cry or dog barking to improve audio experiences in video conferences.
2.1.14. Speech Recognition
Speech recognition, also known as speech to text, enables recognition and translation of spoken language into text. Example applications of speech recognition include transcription, automatic translation, multimodal interaction, real-time captioning and virtual assistants. Speech recognition improves accessibility of auditory content and makes it possible to interact with such content in a privacy-preserving manner in a textual form. Examples of common use cases include watching videos or participating in online meetings using real-time captioning. Models such as [Whisper] approach humans in their accuracy and robustness and are well positioned to improve accessibility of such use cases.
2.1.15. Text Generation
Various text generation use cases are enabled by large language models (LLM) that are able to perform tasks where a general ability to predict the next item in a text sequence is required. This class of models can translate texts, answer questions based on a text input, summarize a larger body of text, or generate text output based on a textual input. LLMs enable better performance compared to older models based on RNN, CNN, or LSTM architectures and further improve the performance of many other use cases discussed in this section. Examples of LLMs include [t5-small], [m2m100_418M], [gpt2], and [llama-2-7b].
2.1.16. Detecting fake video
A user is exposed to realistic fake videos generated by ‘deepfake’ on the web. The fake video can swap the speaker’s face into the president’s face to incite a user politically or to manipulate user’s opinion. The deepfake detection applications such as [FaceForensics++] analyze the videos and protect a user against the fake videos or images. When she watches a fake video on the web, the detection application alerts her of the fraud video in real-time.
2.2. Framework Use Cases
This section collects framework-level use cases for a dedicated low-level API for neural network inference hardware acceleration. It is expected that Machine Learning frameworks will be key consumers of the Web Neural Network API (WebNN API) and the low-level details exposed through the WebNN API are abstracted out from typical web developers. However, it is also expected that web developers with specific interest and competence in Machine Learning will want to interface with the WebNN API directly instead of a higher-level ML framework.
2.2.1. Custom Layer
A web application developer wants to run a DNN model on the WebNN API. However, she has found that some of activation functions like [LeakyReLU], [ELU], etc. are not included in the WebNN API. To address this issue, she constructs custom layers of the additional activation functions on top of the WebNN API. Note that the scope of custom layers may include convolution, normalization, etc. as well as activation.
2.2.2. Network Concatenation
A web application uses a DNN model, and its model data of upper convolutional layers and lower fully-connected layers are stored in separate files, since model data of the fully-connected layers are periodically updated due to fine tuning at the server side.
Therefore, the application downloads both partial model files at first and concatenates them into a single model. When the model is updated, the application downloads fine-tuned part of the model and replace only the fully-connected layers with it.
2.2.3. Performance Adaptation
A web application developer has a concern about performance of her DNN model on mobile devices. She has confirmed that it may run too slow on mobile devices which do not have GPU acceleration. To address this issue, her web application refers to the WebNN API to confirm whether acceleration is available or not, so that the application can display the warning for devices without acceleration.
After several weeks, she has developed a tiny DNN model that can even run on CPU. In order to accommodate CPU execution, she modifies the application so that the application loads the tiny model in the case of CPU-only devices.
2.2.4. Operation Level Execution
A JavaScript ML framework is responsible for loading, interpreting and executing a ML model. During the model execution phase, the framework iterates through the operations of the model and executes each operation on the hardware device, like CPU, GPU or ML accelerator. To avoid the unnecessary data copying across devices, the framework selects the same device to execute the operations. For a compute intensive operation, such as convolution 2D or matrix multiplication, the framework uses WebNN API to execute it with the ML-specific acceleration available on that selected device.
2.2.5. Integration with real-time video processing
The user experience of WebRTC-based video conferencing is enhanced using real-time video processing. For example, background blur implemented using a § 2.1.2 Semantic Segmentation model blurs the background in the user’s live camera feed. To satisfy the performance requirements of this use case, the WebNN API integrates with primitives from other Web APIs that make up the media pipeline to allow WebNN API-based transformation of real-time video streams.
3. Security Considerations
This specification defines a low-level API for neural network inference hardware acceleration. This API is considered a powerful feature [POWERFUL-FEATURES] because it grants low-level access to a user’s computer. To meet the authentication and confidentiality expectations of a powerful feature and to prevent man-in-the-middle attacks, all interfaces defined by this specification are only available in a secure context.This API is disabled by default in all cross-origin frames using the § 6.4 Permissions Policy Integration. This prevents third-party content from using this API unless the embedding page explicitly sets a policy that grants permission.
This API allows creation of an MLContext
from a GPUDevice
defined by WebGPU specification. See WebGPU Security Considerations for more information regarding security characteristics of this context.
Once the graph is fully constructed and compiled, the input shapes into each of the operations in the graph are inferred and finalized. The bounds checking occurs when the compute method is invoked that executes the graph against the actual data. No actual data is bound to the compiled graph before this stage. It is the implementation’s responsibility to make sure proper bounds checking occurs against the shapes of the data already inferred by that time.
Document operations susceptible to out-of-bounds access as a guidance to implementers.
Implementations must defend against control-flow attacks based on changes to data considered to be constant. For example, optimizations in the underlying platform may assume that a weight remains unchanged throughout a computation. If the API allowed the contents of buffers holding weights to change during a computation then those optimization assumptions would be invalidated, causing undefined behavior in the underlying platform. The API mitigates this category of attacks from script by always copying or transferring buffers, but implementations should consider additional defenses such as process isolation of data assumed to be constant.
As a future-proofing measure, the API design allows certain operations that can be generically emulated to be deprecated for security, performance, or other reasons without breaking compatibility. This is made possible by high-level functions that are defined in terms of smaller primitive operations defined in this specifications. This enables a native implementation of a high-level function to be replaced with a polyfill implementation.
Investigate side channel attack feasibility considering the current state where CPU is shared between processes running renderers.
In order to not allow an attacker to target a specific implementation that may contain a flaw, the § 6.2 Device Selection mechanism is a hint only, and the concrete device selection is left to the implementation - a user agent could for instance choose never to run a model on a device with known vulnerabilities. As a further mitigation, no device enumeration mechanism is defined.
Hinting partially mitigates the concern. Investigate additional mitigations.
The API design minimizes the attack surface for the compiled computational graph. The MLGraphBuilder
interface that hosts the various operations is a data definition API and as such doesn’t execute anything, only constructs data. What follows, is that the potential for an attack is limited to when binding the data to the graph before executing it by invoking the MLContext
.compute()
method. This enables implementers to focus on hardening the MLContext
.compute()
method. For example, by making sure it honors the boundary of data and fails appropriately when the bounds are not respected.
Purpose-built Web APIs for measuring high-resolution time mitigate against timing attacks using techniques such as resolution reduction, adding jitter, detection of abuse and API call throttling [hr-time-3]. The practical deployment of WebNN implementations are likely to bring enough jitter to make timing attacks impractical (e.g. because they would use IPC) but implementers are advised to consider and test their implementations against timing attacks.
3.1. Guidelines for new operations
To ensure operations defined in this specification are shaped in a way they can be implemented securely, this section includes guidelines on how operations are expected to be defined to reduce potential for implementation problems. These guidelines are expected to evolve over time to align with industry best practices:
-
Prefer simplicity of arguments
-
Don’t use parsers for complex data formats
-
If an operation can be decomposed to low level primitives:
-
Add an informative emulation path
-
Prefer primitives over new high level operations but consider performance consequences
-
-
Operations should follow a consistent style for inputs and attributes
-
Operation families such as pooling and reduction should share API shape and options
-
Formalize failure cases into test cases whenever possible
-
When in doubt, leave it out: API surface should be as small as possible required to satisfy the use cases, but no smaller
-
Try to keep the API free of implementation details that might inhibit future evolution, do not overspecify
-
Fail fast: the sooner the web developer is informed of an issue, the better
In general, always consider the security and privacy implications as documented in [security-privacy-questionnaire] by the Technical Architecture Group and the Privacy Interest Group when adding new features.
4. Privacy Considerations
This API enhances privacy compared to cloud-based inference, since input data such as locally sourced images or video streams stay within the browser’s sandbox.
This API exposes the minimum amount of information necessary to address the identified § 2 Use cases for the best performance and reliability of results.
No information from the underlying platform is exposed directly. An execution time analysis may reveal indirectly the performance of the underlying platform’s neural network hardware acceleration capabilities relative to another underlying platform.
Note: The group is soliciting further input on the proposed execution time analysis fingerprinting vector and will augment this section with more information and mitigations to inform the implementers of this API.
Unlike WebGPU, this API does not intrinsically support custom shader authoring; and as a result is not prone to timing attacks that rely on shader caches, or other persistent data. The API builds upon pre-existing shaders and lower level primitives of the browser or the underlying OS. Web developers who interface with GPUDevice
are expected to be aware of WebGPU compilation cache considerations.
The WebGPU API identifies machine-specific artifacts as a privacy consideration. Similarly, the WebNN API’s compute unit scheduling may under certain circumstances introduce a fingerprint. However, similarly to WebGPU, such fingerprints are identical across most or all of the devices of each vendor, mitigating the concern. Furthermore, software implementations can be used to further eliminate such artifacts.
The WebNN API defines two developer-settable preferences to help inform § 6.2 Device Selection and allow the implementation to better select the most appropriate underlying execution device for the workload. An MLDeviceType
normatively indicates the kind of device and is one of: "cpu"
, "gpu"
, "npu"
. If this type cannot be satisfied, an "OperationError
" DOMException
is thrown, thus this type can in some cases add two bits of entropy to the fingerprint. An MLPowerPreference
indicates preference as related to the power consumption and is considered a hint only and as such does not increase entropy of the fingerprint.
MLContextOptions
is under active development, and the design is expected to change, informed by further implementation experience and new use cases from the wider web community. [Issue #623]
If a future version of this specification introduces support for a new MLDeviceType
that can only support a subset of MLOperandDataType
s, that may introduce a new fingerprint.
In general, implementers of this API are expected to apply WebGPU Privacy Considerations to their implementations where applicable.
5. Ethical Considerations
The Working Group has started documenting ethical issues associated with using Machine Learning on the Web, to help identify what mitigations its normative specifications should take into account. The Working Group publishes and maintains an Ethical Principles for Web Machine Learning document [webmachinelearning-ethics] open to contributions from the wider community via a dedicated GitHub repository.
6. Programming Model
6.1. Overview
At the heart of neural networks is a computational graph of mathematical operations. These operations are the building blocks of modern machine learning technologies in computer vision, natural language processing, and robotics. The WebNN API is a specification for constructing, compiling, and executing computational graphs of neural networks.
The MLGraph
interface represents a compiled computational graph that is immutable (that is, a model).
The MLGraphBuilder
interface serves as a builder (factory) to construct a computational graph (its graph) that is then compiled to create an MLGraph
.
In WebNN, a computational graph is composed of operators which act on data, and are the nodes of the graph. MLOperand
s are a representation of data that flows within the computational graph, and are the edges of the graph. MLOperand
s include a computational graph's input values for inference, constants (including trained weights) used for inference, intermediate values (often referred to as activations) computed during inference, as well as the output values of inference. An operator's input is one or more MLOperand
s. An operator's output is one or more MLOperand
s. Operators have operator-specific parameters that control their behavior, which can include zero or more activation functions.
A key part of the MLGraphBuilder
interface are methods such as gemm()
and relu()
which create an operator which represents the actual operation to perform on the input data when the computation is run, and return a new MLOperand
holding the operator. Methods that create an MLOperand
connect any inputs and activations to the operator. Each method invocation returns a distinct new value, without changing the value of any other MLOperand
.
An operator has a label, a string which may be included in diagnostics such as exception messages. When an operator is created its label is initialized in an implementation-defined manner and may include the passed label
.
Note: Implementations are encouraged to use the label
provided by developers to enhance error messages and improve debuggability, including both synchronous errors during graph construction and for errors that occur during asynchronous build()
or compute()
operations.
At inference time, every MLOperand
will be bound to a tensor (the actual data), which are essentially multidimensional arrays. The representation of the tensors is implementation dependent, but it typically includes the array data stored in some buffer (memory) and some metadata describing the array data (such as its shape).
Operations within the computational graph have functional semantics. This allows the implementation to potentially share the array data between multiple tensors. For example, the implementation of operations such as reshape, or slice may return a view of its input tensor that shares the same buffer as the input tensor. (In the case of reshape, the entire data is shared, while in the case of slice, a part of the input data is shared.) The implementation may use views, as above, for intermediate values.
Before the execution, the computation graph that is used to compute one or more specified outputs needs to be converted, compiled, and optimized. The key purpose of the compilation step is to enable optimizations that span two or more operations, such as operation or loop fusion. The user agent may also perform these optimizations during graph conversion.
The MLGraphBuilder
.build()
method compiles the graph in the background without blocking the calling thread, and returns a Promise
that resolves to an MLGraph
. Each MLGraphBuilder
can build at most one MLGraph
.
The MLGraph
underlying implementation will be composed of platform-specific representations of operators and operands which correspond to the MLGraphBuilder
's operators and MLOperand
s, but which are not script-visible and may be compositions or decompositions of the graph as constructed by script.
Once the MLGraph
is constructed, the MLContext
.compute()
method performs the execution of the graph asynchronously either on a parallel timeline in a separate worker thread for the CPU execution or on a GPU timeline in a GPU command queue. This method returns immediately without blocking the calling thread while the actual execution is offloaded to a different timeline. The caller supplies the input values using MLNamedArrayBufferViews
, binding the input MLOperand
s to their values. The caller then supplies pre-allocated buffers for output MLOperand
s using MLNamedArrayBufferViews
. The execution produces the results of the computation from all the inputs bound to the graph. The computation results will be placed at the bound outputs at the time the operation is successfully completed on the offloaded timeline at which time the calling thread is signaled. This type of execution supports both the CPU and GPU device.
6.2. Device Selection
An MLContext
interface represents a global state of neural network execution. One of the important context states is the underlying execution device that manages the resources and facilitates the compilation and the eventual execution of the neural network graph. In addition to the default method of creation with MLContextOptions
, an MLContext
could also be created from a specific GPUDevice
that is already in use by the application.
In a situation when a GPU context executes a graph with a constant or an input in the system memory as an ArrayBufferView
, the input content is automatically uploaded from the system memory to the GPU memory, and downloaded back to the system memory of an ArrayBufferView
output buffer at the end of the graph execution. This data upload and download cycles will only occur whenever the execution device requires the data to be copied out of and back into the system memory, such as in the case of the GPU. It doesn’t occur when the device is a CPU device. Additionally, the result of the graph execution is in a known layout format. While the execution may be optimized for a native memory access pattern in an intermediate result within the graph, the output of the last operation of the graph must convert the content back to a known layout format at the end of the graph in order to maintain the expected behavior from the caller’s perspective.
When an MLContext
is created with MLContextOptions
, the user agent selects and creates the underlying execution device by taking into account the application’s MLPowerPreference
and MLDeviceType
options.
6.3. Task Source
The ML task source is a task source to be used for all tasks related to asynchronous compilation and execution of MLGraph
s and creation of MLContext
s.
To queue an ML task given a global object global and a series of steps steps, queue a global task on the ML task source with global and steps.
6.4. Permissions Policy Integration
This specification defines a policy-controlled feature identified by the
string "webnn
".
Its default allowlist is 'self'
.
7. API
7.1. The navigator.ml interface
An ML
object is available in the Window
and DedicatedWorkerGlobalScope
contexts through the Navigator
and WorkerNavigator
interfaces respectively and is exposed via navigator.ml
.
interface mixin { [
NavigatorML SecureContext ,SameObject ]readonly attribute ML ; };
ml Navigator includes NavigatorML ;WorkerNavigator includes NavigatorML ;
7.2. ML
interface
enum MLDeviceType {"cpu" ,"gpu" ,"npu" };enum MLPowerPreference {"default" ,"high-performance" ,"low-power" };dictionary {
MLContextOptions MLDeviceType deviceType = "cpu";MLPowerPreference powerPreference = "default"; }; [SecureContext ,Exposed =(Window ,DedicatedWorker )]interface {
ML Promise <MLContext >createContext (optional MLContextOptions options = {});Promise <MLContext >createContext (GPUDevice gpuDevice ); };
7.2.1. MLContextOptions
MLContextOptions
is under active development, and the design is expected to change, informed by further implementation experience and new use cases from the wider web community. The Working Group is considering additional API controls to allow the definition of a fallback device, multiple devices in a preferred order, or an exclusion of a specific device. Other considerations under discussion include error handling, ultimate fallback, and quantized operators. Feedback is welcome on any of these design considerations from web developers, library authors, OS and hardware vendors, and other stakeholders via GitHub: [Issue #623]
The deviceType
option is an MLDeviceType
and indicates the application’s preference for the kind of device used for the context. It is one of the following:
- "
cpu
" - Provides the broadest compatibility and usability across all client devices with varying degrees of performance.
- "
gpu
" - Provides the broadest range of achievable performance across graphics hardware platforms from consumer devices to professional workstations. The underlying platform implementation may fall back to other devices for certain operators and parts of the graph.
- "
npu
" - Provides power efficiency for sustained workloads across hardware platforms with purpose-built accelerators. The underlying platform implementation may fall back to other devices for certain operators and parts of the graph.
The powerPreference
option is an MLPowerPreference
and indicates the application’s preference as related to power consumption. It is one of the following:
- "
default
" - Let the user agent select the most suitable behavior.
- "
high-performance
" - Prioritizes execution speed over power consumption.
- "
low-power
" - Prioritizes power consumption over other considerations such as execution speed.
7.2.2. createContext()
-
options
: anMLContextOptions
. Provides the application’s preferences for the context. -
gpuDevice
: aGPUDevice
. A specific device to use with the context.
MLContext
.
To create a context given realm realm and options (a GPUDevice
or MLContextOptions
), run these steps:
-
Let context be a new
MLContext
object with realm. -
If options is a
GPUDevice
object:-
Set context.
[[contextType]]
to "webgpu". -
Set context.
[[deviceType]]
to"gpu"
. -
Set context.
[[powerPreference]]
to"default"
.
-
-
Otherwise:
-
Set context.
[[contextType]]
to "default". -
If options["
deviceType
"] exists, then set context.[[deviceType]]
to options["deviceType
"]. Otherwise, set context.[[deviceType]]
to"cpu"
. -
If options["
powerPreference
"] exists, then set context.[[powerPreference]]
to options["powerPreference
"]. Otherwise, set context.[[powerPreference]]
to"default"
.
-
-
If the user agent cannot support context.
[[contextType]]
, context.[[deviceType]]
and context.[[powerPreference]]
, return failure. -
Return context.
The createContext(options)
steps are:
-
Let global be this's relevant global object.
-
If global’s associated Document is not allowed to use the webnn feature, return a new promise rejected with a "
SecurityError
"DOMException
. -
Let realm be this's relevant realm.
-
Let promise be a new promise.
-
Run the following steps in parallel.
-
Let context be the result of creating a context given realm and options. If that returns failure, then queue an ML task with global to reject promise with a "
NotSupportedError
"DOMException
and abort these steps. -
Queue an ML task with global to resolve promise with context.
-
-
Return promise.
The createContext(gpuDevice)
method steps are:
-
Let global be this's relevant global object.
-
If global’s associated Document is not allowed to use the webnn feature, return a new promise rejected with a "
SecurityError
"DOMException
. -
Let realm be this's relevant realm.
-
Let promise be a new promise.
-
Run the following steps in parallel.
-
Let context be the result of creating a context given realm and gpuDevice. If that returns failure, then queue an ML task with global to reject promise with a "
NotSupportedError
"DOMException
and abort these steps. -
Queue an ML task with global to resolve promise with context.
-
-
Return promise.
7.3. MLContext
interface
The MLContext
interface represents a global state of neural network compute workload and execution processes. Each MLContext
object has associated context type, MLDeviceType
and MLPowerPreference
.
typedef record <USVString ,ArrayBufferView >;
MLNamedArrayBufferViews dictionary {
MLComputeResult MLNamedArrayBufferViews inputs ;MLNamedArrayBufferViews outputs ; }; [SecureContext ,Exposed =(Window ,DedicatedWorker )]interface {
MLContext Promise <MLComputeResult >compute (MLGraph graph ,MLNamedArrayBufferViews inputs ,MLNamedArrayBufferViews outputs );MLOpSupportLimits (); };
opSupportLimits
MLContext
has the following internal slots:
[[contextType]]
of type context type.-
The
MLContext
's context type. [[deviceType]]
of typeMLDeviceType
.-
The
MLContext
'sMLDeviceType
. [[powerPreference]]
of typeMLPowerPreference
.-
The
MLContext
'sMLPowerPreference
.
The context type is the type of the execution context that manages the resources and facilitates the compilation and execution of the neural network graph:
- "default"
- Context created per user preference options.
- "webgpu"
- Context created from WebGPU device.
[[contextType]]
is set to default with the MLContextOptions
.deviceType
set to "gpu"
, the user agent is responsible for creating an internal GPU device that operates within the context and is capable of ML workload submission on behalf of the calling application. In this setting however, only ArrayBufferView
inputs and outputs are allowed in and out of the graph execution since the application has no way to know what type of internal GPU device is being created on their behalf. In this case, the user agent is responsible for automatic uploads and downloads of the inputs and outputs to and from the GPU memory using this said internal device. inputs
, of type MLNamedArrayBufferViews-
An object where the keys are the graph input names, and the values are the transferred
ArrayBufferView
s for the supplied input tensor values. outputs
, of type MLNamedArrayBufferViews-
An object where the keys are the graph output names, and the values are the transferred
ArrayBufferView
s for the computed output tensor values.
To validate buffer with descriptor given ArrayBufferView
bufferView and MLOperandDescriptor
descriptor, run the following steps:
-
If bufferView’s element type does not match to descriptor.
dataType
according to this table, return false. -
If bufferView.[[ByteLength]] is not equal to descriptor’s byte length, return false.
To execute graph, given MLGraph
graph, MLNamedArrayBufferViews
inputs and MLNamedArrayBufferViews
outputs, run the following steps. They return undefined
, or an error.
-
Let inputResources be the input resources of graph.
[[implementation]]
. -
For each name → inputValue of inputs:
-
Let inputDescriptor be graph.
[[inputDescriptors]]
[name]. -
Let inputTensor be a new tensor for graph.
[[implementation]]
as follows:-
Set the data type of inputTensor to the one that matches inputValue’s element type.
-
Set the shape of inputTensor to inputDescriptor.
shape
. -
Set the values of elements in inputTensor to the values of elements in inputValue.
-
-
Request the underlying implementation of graph to bind inputResources[name] to inputTensor.
-
-
For each name → outputValue of outputs:
-
Issue a compute request to graph.
[[implementation]]
given name and inputResources and wait for completion.-
If that returns an error, then return an "
OperationError
"DOMException
. -
Otherwise, let outputTensor be the result.
-
-
Let outputDesc be graph.
[[outputDescriptors]]
[name]. -
If the byte length of outputTensor is not equal to outputDesc’s byte length, then return a
TypeError
. -
If outputTensor’s element type doesn’t match outputValue’s element type, then return a
TypeError
. -
Request the underlying implementation of graph to set the values of elements in outputValue to the values of elements in outputTensor.
-
-
Return
undefined
.
7.3.1. MLNamedArrayBufferViews
transfer algorithm
To transfer an MLNamedArrayBufferViews
views with realm realm:
-
For each name → view of views:
-
If view is not transferable, then throw a
TypeError
.
-
-
Let transferredViews be a new
MLNamedArrayBufferViews
. -
For each name → view of views:
-
Let transferredBuffer be the result of transferring view’s underlying buffer.
-
Assert: The above step never throws an exception.
-
Let constructor be the appropriate view constructor for the type of
ArrayBufferView
view from realm. -
Let elementsNumber be the result of view’s byte length / view’s element size.
-
Let transferredView be Construct(constructor, transferredBuffer, view.[[ByteOffset]], elementsNumber).
-
Set transferredViews[name] to transferredView.
-
-
Return transferredViews.
7.3.2. compute()
Asynchronously carries out the computational workload of a compiled graph MLGraph
on a separate timeline, either on a worker thread for the CPU execution, or on a GPU/NPU timeline for submitting a workload onto the command queue. The asynchronous nature of this call avoids blocking the calling thread while the computation for result is ongoing. This method of execution requires an MLContext
created with MLContextOptions
. Otherwise, it throws an "OperationError
" DOMException
.
MLNamedArrayBufferViews
to new views that share the same backing memory allocations. The transferred views are returned to the caller via the promise fulfillment with the computation result written into the backing memory of the output views. -
graph
: anMLGraph
. The compiled graph to be executed. -
inputs
: anMLNamedArrayBufferViews
. The resources of inputs. Will be transferred if there are no validation errors. -
outputs
: anMLNamedArrayBufferViews
. The pre-allocated resources of required outputs. Will be transferred if there are no validation errors.
Returns: Promise
<MLComputeResult
>.
Note: Invocations of compute()
will fail if any of the graph
's inputs are not provided as inputs
, or if any requested outputs
do not match the graph
's outputs.
The compute(graph, inputs, outputs)
method steps are:
-
Let global be this's relevant global object.
-
Let realm be this's relevant realm.
-
If graph.
[[context]]
is not this, then return a new promise rejected with aTypeError
. -
If graph.
[[context]]
.[[contextType]]
is not "default", then return a new promise rejected with an "OperationError
"DOMException
. -
For each name → descriptor of graph.
[[inputDescriptors]]
:-
If inputs[name] does not exist, then return a new promise rejected with a
TypeError
. -
If validating buffer with descriptor given inputs[name] and descriptor returns false, then return a new promise rejected with a
TypeError
.
-
-
For each name → resource of outputs:
-
If graph.
[[outputDescriptors]]
[name] does not exist, then return a new promise rejected with aTypeError
. -
If validating buffer with descriptor given resource and graph.
[[outputDescriptors]]
[name] returns false, then return a new promise rejected with aTypeError
.
-
-
Let transferredInputs be the result of transferring
MLNamedArrayBufferViews
inputs with realm. If that threw an exception, then return a new promise rejected with that exception. -
Let transferredOutputs be the result of transferring
MLNamedArrayBufferViews
outputs with realm. If that threw an exception, then return a new promise rejected with that exception. -
Let promise be a new promise.
-
Run the following steps in parallel:
-
Invoke execute graph given graph, transferredInputs and transferredOutputs. If that returns an error, then queue an ML task with global to reject promise with an equivalent error in realm and abort these steps.
-
Let result be a new
MLComputeResult
with realm. -
Set result.
inputs
to transferredInputs. -
Set result.
outputs
to transferredOutputs. -
Queue an ML task with global to resolve promise with result.
-
-
Return promise.
7.3.2.1. Examples
The following code showcases the asynchronous computation.
const operandType= { dataType: 'float32' , shape: [ 2 , 2 ] }; const context= await navigator. ml. createContext(); const builder= new MLGraphBuilder( context); // 1. Create a computational graph 'C = 0.2 * A + B'. const constant= builder. constant( operandType. dataType, 0.2 ); const A= builder. input( 'A' , operandType); const B= builder. input( 'B' , operandType); const C= builder. add( builder. mul( A, constant), B); // 2. Compile it into an executable. const graph= await builder. build({ 'C' : C}); // 3. Bind inputs to the graph and execute for the result. const bufferA= new Float32Array( 4 ). fill( 1.0 ); const bufferB= new Float32Array( 4 ). fill( 0.8 ); const bufferC= new Float32Array( 4 ); const inputs= { 'A' : bufferA, 'B' : bufferB}; const outputs= { 'C' : bufferC}; const result= await context. compute( graph, inputs, outputs); // The computed result of [[1, 1], [1, 1]] is in the buffer associated with // the output operand. console. log( 'Output value: ' + result. outputs. C); // Note: the result.outputs.C buffer is different from the bufferC, but it // shares the same backing memory allocation.
7.3.3. opSupportLimits()
The opSupportLimits()
exposes level of support that differs across implementations at operator level. Consumers of the WebNN API are encouraged to probe feature support level by using opSupportLimits()
to determine the optimal model architecture to be deployed for each target platform.
7.3.3.1. MLOpSupportLimits
dictionary
The MLOpSupportLimits
has following top level members, aside from these, each operator has a corresponding member defined in its builder method.
dictionary {
MLOpSupportLimits MLInputOperandLayout preferredInputLayout ;MLSupportLimits input ;MLSupportLimits constant ;MLSupportLimits output ; };
preferredInputLayout
, of type MLInputOperandLayout-
Preferred input layout for layout dependent operators like
conv2d()
. input
, of type MLSupportLimitsconstant
, of type MLSupportLimitsoutput
, of type MLSupportLimits
7.3.3.2. MLSupportLimits
dictionary
dictionary {
MLSupportLimits sequence <MLOperandDataType >dataTypes ; };
dataTypes
, of type sequence<MLOperandDataType>-
Supported data types.
7.3.3.3. MLBinarySupportLimits
dictionary
dictionary {
MLBinarySupportLimits MLSupportLimits a ;MLSupportLimits b ;MLSupportLimits output ; };
a
, of type MLSupportLimits-
MLSupportLimits
for a operand. b
, of type MLSupportLimits-
MLSupportLimits
for b operand. output
, of type MLSupportLimits-
MLSupportLimits
for output operand.
7.3.3.4. MLSingleInputSupportLimits
dictionary
dictionary {
MLSingleInputSupportLimits MLSupportLimits input ;MLSupportLimits output ; };
input
, of type MLSupportLimits-
MLSupportLimits
for input operand. output
, of type MLSupportLimits-
MLSupportLimits
for output operand.
7.4. MLGraph
interface
The MLGraph
interface represents a compiled computational graph. A compiled graph once constructed is immutable and cannot be subsequently changed.
[SecureContext ,Exposed =(Window ,DedicatedWorker )]interface {};
MLGraph
MLGraph
has the following internal slots:
[[context]]
of typeMLContext
[[inputDescriptors]]
of type record<USVString
,MLOperandDescriptor
>-
Maps the name of an input
MLOperand
to itsMLOperandDescriptor
for all inputMLOperand
s of thisMLGraph
. [[outputDescriptors]]
of type record<USVString
,MLOperandDescriptor
>-
Maps the name of an output
MLOperand
to itsMLOperandDescriptor
for all outputMLOperand
s of thisMLGraph
. [[implementation]]
-
The underlying implementation provided by the User Agent.
7.5. MLOperandDescriptor
dictionary
An MLOperandDescriptor
describes the shape (dimensions) and data type of an operand. They are used to describe the inputs and constants for an MLGraph
, and every MLOperand
has an internal MLOperandDescriptor
.
enum {
MLInputOperandLayout ,
"nchw" };
"nhwc" enum {
MLOperandDataType ,
"float32" ,
"float16" ,
"int32" ,
"uint32" ,
"int64" ,
"uint64" ,
"int8" };
"uint8" dictionary {
MLOperandDescriptor required MLOperandDataType dataType ;required sequence <[EnforceRange ]unsigned long >shape ; };
dataType
, of type MLOperandDataType-
The operand data type.
shape
, of typesequence<[EnforceRange] unsigned long>
-
The list of dimensions of the operand. It is empty for scalar operands.
To create an MLOperandDescriptor
given MLOperandDataType
dataType and list shape, run the following steps:
-
Let descriptor be a new
MLOperandDescriptor
. -
Set descriptor.
dataType
to dataType. -
Return descriptor.
The byte length of an MLOperandDescriptor
desc is the value returned by the following steps:
-
Let elementLength be 1.
-
For each dimension of desc.
shape
:-
Set elementLength to elementLength * dimension.
-
-
Let elementSize be the element size of one of the
ArrayBufferView
types that matches desc.dataType
according to this table. -
Return elementLength * elementSize.
A valid dimension is an integer greater than zero and in the range of long
. Implementations may impose a smaller upper bound.
Should 0-size dimensions be supported? [Issue #391]
To check dimensions given MLOperandDescriptor
descriptor, run the following steps:
-
If any element of descriptor.
shape
is not a valid dimension, return false. -
If descriptor.
shape
's size is too large to be supported by the implementation, return false.The maximum number of operand dimensions is not defined, but native ML APIs usually have a maximum supported size. [Issue #456]
-
If descriptor’s byte length is not supported by the implementation, then return false.
-
Return true.
7.6. MLOperand
interface
An MLOperand
represents an intermediary graph being constructed as a result of compositing parts of an operation into a fully composed operation.
For instance, an MLOperand
may represent a constant feeding to an operation or the result from combining multiple constants together into an operation. See also § 6 Programming Model.
[SecureContext ,Exposed =(Window ,DedicatedWorker )]interface {
MLOperand MLOperandDataType dataType ();sequence <unsigned long >shape (); };dictionary {
MLOperatorOptions USVString label = ""; };typedef (bigint or unrestricted double )MLNumber ;
MLOperand
has the following internal slots:
[[builder]]
of typeMLGraphBuilder
-
The
MLOperand
's associated builder object. [[descriptor]]
of typeMLOperandDescriptor
-
The
MLOperand
's descriptor. [[name]]
of type string-
The
MLOperand
's name (only for input operands). [[operator]]
of type operator
An MLOperand
's shape is its [[descriptor]]
.shape
.
An MLOperand
's rank is its shape's size.
An MLOperand
's dataType is its [[descriptor]]
.dataType
.
Since the [[builder]]
object is bound by the MLGraphBuilder()
constructor to an MLContext
object, an MLOperand
is also always bound to the same MLContext
object.
MLOperatorOptions
has the following members:
label
, of type USVString, defaulting to""
-
Optionally provided when an operator is created using
MLGraphBuilder
methods that createMLOperand
s. The implementation may use this value to initialize the operator's label.
7.6.1. Creating an MLOperand
The MLOperand
objects are created by the methods of MLGraphBuilder
, internally using the following algorithms.
To create an MLOperand given MLGraphBuilder
builder and MLOperandDescriptor
desc, run the following steps:
-
Let operand be a new
MLOperand
. -
Set operand.
[[builder]]
to builder. -
Set operand.
[[descriptor]]
to desc. -
Return operand.
To copy an MLOperand given MLOperand
operand, run the following steps:
-
Let result be a new
MLOperand
. -
Set result.
[[builder]]
to operand.[[builder]]
. -
Set result.
[[descriptor]]
to operand.[[descriptor]]
. -
If operand.
[[name]]
exists, then set result.[[name]]
to operand.[[name]]
. -
Return result.
To validate operand given MLGraphBuilder
builder and MLOperand
operand, return true if operand.[[builder]]
is builder, and false otherwise.
7.6.1.1. MLNumber
MLNumber
is used when specifying the type of a numeric option for an MLOperand
which can be of any MLOperandDataType
, including both 64-bit integer types ("uint64"
and "int64"
) and 32-bit floating point ("float32"
). Implementations process the value according to the corresponding MLOperandDataType
. For example, if clamp(input, options)
is called with an MLOperand
with dataType "uint32"
, the MLNumber
parameters are explicitly cast to unsigned long
.
double
would lose accuracy when passing values over 253, and specifying long long
would disallow values over 263. Support for unions of bigint
and numeric types is new in [WEBIDL], and implementation support is also limited. Prototype implementations are encouraged to provide feedback for this approach. [Issue #whatwg/webidl#1388]
7.6.2. dataType()
Returns the data type of the MLOperand
.
MLOperandDataType
. The data type of the operand. 7.6.3. shape()
Returns the shape of the MLOperand
.
unsigned long
>. The shape of the operand. 7.7. MLGraphBuilder
interface
The MLGraphBuilder
interface defines a set of operations as identified by the § 2 Use cases that can be composed into a computational graph. It also represents the intermediate state of a graph building session.
typedef record <USVString ,MLOperand >; [
MLNamedOperands SecureContext ,Exposed =(Window ,DedicatedWorker )]interface { // Construct the graph builder from the context.
MLGraphBuilder constructor (MLContext context ); // Create an operand for a graph input.MLOperand input (USVString name ,MLOperandDescriptor descriptor ); // Create an operand for a graph constant.MLOperand constant (MLOperandDescriptor descriptor ,ArrayBufferView bufferView ); // Create a scalar operand from the specified number of the specified type.MLOperand constant (MLOperandDataType type ,MLNumber value ); // Compile the graph up to the specified output operands asynchronously.Promise <MLGraph >build (MLNamedOperands outputs ); };
MLGraphBuilder
.build()
method compiles the graph builder state up to the specified output operands into a compiled graph according to the type of MLContext
that creates it. When the [[contextType]]
of the MLContext
is set to "default", the compiled graph is initialized right before the MLGraph
is returned. This graph initialization stage is important for optimal performance of the subsequent graph executions. It typically involves a process known as "weight preprocessing" where all the constant inputs to the graph are preprocessed and cached at the operating system level for subsequent graph execution calls. The initializing inputs are typically the constant weight data specified through the constant()
method as constant operands during graph construction time. MLGraphBuilder
has the following internal slots:
[[context]]
of typeMLContext
-
The context of type
MLContext
associated with thisMLGraphBuilder
. [[hasBuilt]]
of typeboolean
-
Whether
MLGraphBuilder
.build()
has been called. Once built, theMLGraphBuilder
can no longer create operators or compileMLGraph
s.
7.7.1. MLGraphBuilder
constructor
-
context
: anMLContext
. The context to associate with theMLGraphBuilder
.
The new MLGraphBuilder(context)
constructor steps are:
-
If this's relevant global object's associated Document is not allowed to use the webnn feature, then throw a "
SecurityError
"DOMException
. -
Set this.
[[context]]
to context. -
Set this.
[[hasBuilt]]
to false.
7.7.2. input operands
Create a named MLOperand
based on a descriptor, that can be used as an input.
-
name
: a string name of the input. -
descriptor
: anMLOperandDescriptor
object.
MLOperand
.
The input(name, descriptor)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If any
MLOperand
s in this's graph's inputs have a[[name]]
equal to name, then throw aTypeError
. -
If checking dimensions given descriptor returns false, then throw a
TypeError
. -
Make graph connections:
-
Return operand.
MLGraphBuilder
API allows creating an MLGraph
without input operands. If the underlying platform doesn’t support that, implementations may add a stub input, or pass constants as inputs to the graph. 7.7.3. constant operands
Create a constantMLOperand
that can be used in MLGraphBuilder
methods.
7.7.3.1. constant(descriptor, bufferView)
Create a constant MLOperand
of the specified data type and shape that contains the initializing data.
-
descriptor
: anMLOperandDescriptor
. The descriptor of the output tensor. -
bufferView
: anArrayBufferView
. The view of the buffer containing the initializing data.
MLOperand
. The constant output tensor.
The constant(descriptor, bufferView)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If checking dimensions given descriptor returns false, then throw a
TypeError
. -
If validating buffer with descriptor given bufferView and descriptor returns false, then throw a
TypeError
. -
Make graph connections:
-
Let operand be the result of creating an MLOperand given this and descriptor.
-
Let bytes be the result of getting a copy of the bytes held by the buffer source given bufferView.
-
Add operand to this's graph's constants with bytes as value.
-
-
Return operand.
7.7.3.2. constant(type, value)
Create a scalar constant MLOperand
of the specified value and data type.
"int8"
data type, etc. -
type
: anMLOperandDataType
. -
value
: anMLNumber
. The value of the constant.
MLOperand
. The constant output.
The constant(type, value)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
Set value to the result of casting value to type.
-
Let descriptor be the result of creating an MLOperandDescriptor given type and « ».
-
Make graph connections:
-
Let operand be the result of creating an MLOperand given this and descriptor.
-
Add operand to this's graph's constants with value as value.
-
-
Return operand.
7.7.4. build method
Build a composed graph up to a given output operand into a computational graph asynchronously.-
outputs
: anMLNamedOperands
. Identifies theMLOperand
s that will be the outputs of the graph.
MLGraph
>.
The build(outputs)
method steps are:
-
If this.
[[hasBuilt]]
is true, then return a new promise rejected with an "InvalidStateError
"DOMException
. -
If outputs is empty, then return a new promise rejected with a
TypeError
. -
For each name → operand of outputs:
-
If name is empty, then return a new promise rejected with a
TypeError
. -
If validating operand given this and operand returns false, then return a new promise rejected with a
TypeError
. -
If operand is in this's graph's inputs or constants, then return a new promise rejected with a
TypeError
.
-
-
Let operands be a new empty set.
-
Let operators be a new empty set.
-
Let inputs be a new empty set.
-
While queue is not empty:
-
Let global be this's relevant global object.
-
Let realm be this's relevant realm.
-
Let graph be a new
MLGraph
with realm. -
Set graph.
[[context]]
to this.[[context]]
. -
For each operand in inputs:
-
Set graph.
[[inputDescriptors]]
[operand.[[name]]
] to operand.[[descriptor]]
.If
constants'
ArrayBuffer
s are not transferred, make copies for graph's constants here. [Issue #566]
-
-
For each name → operand of outputs:
-
Set graph.
[[outputDescriptors]]
[name] to operand.[[descriptor]]
.
-
-
Let promise be a new promise.
-
Run the following steps in parallel:
-
Let graphImpl be the result of converting this's graph with operands, operators, inputs, and outputs’s values into an implementation-defined format which can be interpreted by the underlying platform.
-
If the underlying platform does not support a requested feature, then queue an ML task with global to reject promise with an "
OperationError
"DOMException
, and abort these steps.
-
-
Set graph.
[[implementation]]
to graphImpl. -
Queue an ML task with global to resolve promise with graph.
-
-
Set this.
[[hasBuilt]]
to true. -
Return promise.
NOTE: Specifying an input operand or constant operand as a graph output
results in an error, as this is usually an incorrect usage of the API. Callers can work around this by introducing an identity()
operator.
7.7.5. argMin/argMax operations
Return the index location of the minimum or maximum values of all the input values along the axis. In case of ties, the identity of the return value is implementation dependent.dictionary :
MLArgMinMaxOptions MLOperatorOptions {boolean keepDimensions =false ;MLOperandDataType outputDataType = "int32"; };partial interface MLGraphBuilder {MLOperand argMin (MLOperand input , [EnforceRange ]unsigned long axis ,optional MLArgMinMaxOptions options = {});MLOperand argMax (MLOperand input , [EnforceRange ]unsigned long axis ,optional MLArgMinMaxOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits argMin ;MLSingleInputSupportLimits argMax ; };
MLArgMinMaxOptions
has the following members:
keepDimensions
, of type boolean, defaulting tofalse
-
If true, retains reduced dimensions with size 1.
outputDataType
, of type MLOperandDataType, defaulting to"int32"
-
An
MLOperandDataType
. The output data type.
-
input
: anMLOperand
. The input N-D tensor. -
axis
: The dimension to reduce. The value must be in the range [0, N-1] where N is the rank of the input tensor. -
options
: an optionalMLArgMinMaxOptions
. The optional parameters of the operation.
Returns: an MLOperand
. The N-D tensor of the reduced shape. The values must be of type options.outputDataType
in the range [0, N-1] where N is the size of the input dimension specified by axis.
MLOpSupportLimits
has following members for argMin()
and argMax()
:
argMin
, of type MLSingleInputSupportLimits-
Support limits for operator
argMin()
. argMax
, of type MLSingleInputSupportLimits-
Support limits for operator
argMax()
.
To create argMin/argMax operation given string op, MLOperand
input, unsigned long
axis, and MLArgMinMaxOptions
options, run the following steps:
-
Assert: op is one of "argMin", "argMax".
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
If input’s shape[axis] is greater than options.
outputDataType
's maximum value, throw aTypeError
. -
Let outputShape be the result of calculating reduction output sizes given input’s shape, « axis », and options.
keepDimensions
. If that returns failure, then throw aTypeError
. -
Let desc be the result of creating an MLOperandDescriptor given options.
outputDataType
and outputShape. -
Make graph connections:
-
Let operator be an operator for the op operation, given options.
-
Let output be the result of creating an MLOperand given this and desc.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
The following argMin/argMax algorithms are supported.
argMin(input, axis, options)
method steps are:
-
Let output be the result of running the create argMin/argMax operation given "argMin", input, axis and options.
-
Return output.
argMax(input, axis, options)
method steps are:
-
Let output be the result of running the create argMin/argMax operation given "argMax", input, axis and options.
-
Return output.
7.7.6. batchNormalization
Normalize the values of the input tensor using [Batch-Normalization]. For each input feature, the mean and variance values of that feature are computed across all the samples in the batch dimension while the model is trained. These mean and variance values are then subsequently given to this operation during model inference.dictionary :
MLBatchNormalizationOptions MLOperatorOptions {MLOperand scale ;MLOperand bias ; [EnforceRange ]unsigned long axis = 1;double epsilon = 1e-5; };partial interface MLGraphBuilder {MLOperand batchNormalization (MLOperand input ,MLOperand mean ,MLOperand variance ,optional MLBatchNormalizationOptions options = {}); };dictionary {
MLBatchNormalizationSupportLimits MLSupportLimits input ;MLSupportLimits mean ;MLSupportLimits variance ;MLSupportLimits scale ;MLSupportLimits bias ;MLSupportLimits output ; };partial dictionary MLOpSupportLimits {MLBatchNormalizationSupportLimits batchNormalization ; };
MLBatchNormalizationOptions
has the following members:
scale
, of type MLOperand-
The 1-D tensor of the scaling values whose size is equal to the size of the input dimension denoted by
axis
. bias
, of type MLOperand-
The 1-D tensor of the bias values whose size is equal to the size of the input dimension denoted by
axis
. axis
, of type unsigned long, defaulting to1
-
The index to the feature count dimension of the input shape for which the mean and variance values are. Its value must be in the range [0, N-1] where N is the rank of the input tensor. The default value is 1, corresponding to the channel ("c") dimension in the
"nchw"
data layout. epsilon
, of type double, defaulting to1e-5
-
A small value to prevent computational error due to divide-by-zero.
-
input
: anMLOperand
. The input N-D tensor. -
mean
: anMLOperand
. Specifies the 1-D tensor of the mean values of the input features across the batch. Its size is equal to the size of the input dimension denoted byaxis
. -
variance
: anMLOperand
. The 1-D tensor of the variance values of the input features across the batch whose size is equal to the size of the input dimension denoted byaxis
. -
options
: an optionalMLBatchNormalizationOptions
. Specifies the optional parameters of the operation.
Returns: an MLOperand
. The batch-normalized N-D tensor of the same shape as input.
MLBatchNormalizationSupportLimits
has following members:
input
, of type MLSupportLimits-
MLSupportLimits
for input operand. mean
, of type MLSupportLimits-
MLSupportLimits
for mean operand. variance
, of type MLSupportLimits-
MLSupportLimits
for variance operand. scale
, of type MLSupportLimits-
MLSupportLimits
for scale operand. bias
, of type MLSupportLimits-
MLSupportLimits
for bias operand. output
, of type MLSupportLimits-
MLSupportLimits
for output operand.
MLOpSupportLimits
has following members for batchNormalization()
:
batchNormalization
, of type MLBatchNormalizationSupportLimits-
Support limits for operator
batchNormalization()
.
The batchNormalization(input, mean, variance, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of input, mean, variance, options.
scale
(if it exists), and options.bias
(if it exists) returns false, then throw aTypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
If options.
axis
is not in the range 0 to input’s rank, exclusive, then throw aTypeError
. -
If mean’s dataType is not equal to input’s dataType, then throw a
TypeError
. -
If mean’s shape is not equal to « input’s shape[options.
axis
] », then throw aTypeError
. -
If variance’s dataType is not equal to input’s dataType, then throw a
TypeError
. -
If variance’s shape is not equal to « input’s shape[options.
axis
] », then throw aTypeError
. -
Set options.
epsilon
to the result of casting options.epsilon
to input’s dataType. -
Make graph connections:
-
Let operator be an operator for the "batchNormalization" operation, given input, mean, variance and options.
-
Let output be the result of creating an MLOperand given this and input.
[[descriptor]]
. -
Set output.
[[operator]]
to operator. -
Set operator’s inputs to input, mean, and variance.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation when the input tensor is 4-D of the "nchw"
layout can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function batchNormalization( builder, input, mean, variance, options) { const shape= [ 1 , input. shape()[ options. axis], 1 , 1 ]; return builder. add( builder. mul( builder. reshape( options. scale, shape), builder. div( builder. sub( input, builder. reshape( mean, shape)), builder. sqrt( builder. add( builder. reshape( variance, shape), builder. constant( input. dataType(), options. epsilon))))), builder. reshape( options. bias, shape)); }
7.7.7. cast
Cast each element in the input tensor to the target data type.partial interface MLGraphBuilder {MLOperand cast (MLOperand input ,MLOperandDataType type ,optional MLOperatorOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits cast ; };
-
input
: anMLOperand
. The input N-D tensor. -
type
: anMLOperandDataType
. The target data type. -
options
: anMLOperatorOptions
. Specifies the optional parameters of the operation.
Returns: an MLOperand
. The N-D tensor of the same shape as input with each element casted to the target data type.
MLOpSupportLimits
has following members for cast()
:
cast
, of type MLSingleInputSupportLimits-
Support limits for operator
cast()
.
Casting between MLOperandDataType
s is specified for some cases and implementation-defined in other cases, according to the following table:
Target type Input type | "float32" , "float16"
| "int32" , "uint32" , "int64" , "uint64" , "int8" , "uint8"
|
---|---|---|
"float32" , "float16"
|
If in range, nearest representable value.
If out of range, +/-Infinity. |
If in range, truncated.
If out of range, implementation-defined. |
"int32" , "uint32" , "int64" , "uint64" , "int8" , "uint8"
|
If in range, nearest representable value.
If out of range, +/-Infinity. |
If in range, same value.
If out of range, lowest N bits reinterpreted as target type, assuming two’s complement for signed types. |
NOTE: For example, casting -1 from "int8"
to "uint8"
is specified to yield 255. But casting -1 from "float32"
to "uint8"
is implementation-defined.
The cast(input, type, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
Make graph connections:
-
Let operator be an operator for the "cast" operation, given type and options.
-
Let output be the result of copying an MLOperand given input.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
7.7.8. clamp
Clamp the input tensor element-wise within a range specified by the minimum and maximum values.dictionary :
MLClampOptions MLOperatorOptions {MLNumber minValue ;MLNumber maxValue ; };partial interface MLGraphBuilder {MLOperand clamp (MLOperand input ,optional MLClampOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits clamp ; };
MLClampOptions
has the following members:
minValue
, of type MLNumber-
The minimum value of the range. When it is not specified, the clamping is not performed on the lower limit of the range.
maxValue
, of type MLNumber-
The maximum value of the range. When it is not specified, the clamping is not performed on the upper limit of the range.
-
input
: anMLOperand
. The input tensor. -
options
: an optionalMLClampOptions
. The optional parameters of the operation.
-
an
MLOperand
. The output tensor of the same shape as input.
MLOpSupportLimits
has following member for clamp()
:
clamp
, of type MLSingleInputSupportLimits-
Support limits for operator
clamp()
.
The clamp(input, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
Let minValue be the options.
minValue
if given, or Infinity otherwise. -
Set options.
minValue
to the result of casting minValue to input’s dataType. -
Let maxValue be the options.
maxValue
if given, or -Infinity otherwise. -
Set options.
maxValue
to the result of casting maxValue to input’s dataType. -
If options.
minValue
is greater than options.maxValue
, then throw aTypeError
. -
Make graph connections:
-
Let output be the result of copying an MLOperand given input.
-
Let operator be an operator for the "clamp" operation, given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function clamp( builder, input, options) { if ( options. minValue=== undefined ) { if ( options. maxValue=== undefined ) { return input; } else { return builder. min( input, builder. constant( input. dataType(), options. maxValue)); } } else { if ( options. maxValue=== undefined ) { return builder. max( input, builder. constant( input. dataType(), options. minValue)); } else { return builder. min( builder. max( input, builder. constant( input. dataType(), options. minValue)), builder. constant( input. dataType(), options. maxValue)); } } }
7.7.9. concat
Concatenates the input tensors along a given axis.partial interface MLGraphBuilder {MLOperand concat (sequence <MLOperand >inputs , [EnforceRange ]unsigned long axis ,optional MLOperatorOptions options = {}); };dictionary {
MLConcatSupportLimits MLSupportLimits inputs ;MLSupportLimits output ; };partial dictionary MLOpSupportLimits {MLConcatSupportLimits concat ; };
-
inputs
: a sequence<MLOperand
>. All input tensors must have the same shape, except for the size of the dimension to concatenate on. -
axis
: anunsigned long
scalar. The axis that the inputs concatenate along. Its value must be in the range [0, N-1] where N is the rank of the input tensors. -
options
: anMLOperatorOptions
. Specifies the optional parameters of the operation.
Returns: an MLOperand
. The concatenated tensor of all the inputs along
the axis. The output tensor has the same shape except on the dimension
that all the inputs concatenated along. The size of that dimension is
computed as the sum of all the input sizes of the same dimension.
MLConcatSupportLimits
has following members:
inputs
, of type MLSupportLimits-
MLSupportLimits
for all input operands. output
, of type MLSupportLimits-
MLSupportLimits
for output operand.
MLOpSupportLimits
has following member for concat()
:
concat
, of type MLConcatSupportLimits-
Support limits for operator
concat()
.
The concat(inputs, axis, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any item in inputs returns false, then throw a
TypeError
. -
Let first be inputs[0].
-
If axis is greater than or equal to first’s rank, then throw a
TypeError
. -
Let desc be the result of creating an MLOperandDescriptor given first’s dataType and first’s shape.
-
For each index in the range 1 to inputs’s size, exclusive:
-
Let input be inputs[index].
-
If input’s dataType is not equal to first’s dataType, then throw a
TypeError
. -
If input’s rank is not equal to first’s rank, then throw a
TypeError
. -
For each dim in the range 0 to input’s rank, exclusive:
If the shape of each corresponding dimension and type of the operands, except for those of the dimension given by axis, is not the same, fail.
-
-
Make graph connections:
-
Let output be the result of creating an MLOperand given this and desc.
-
Let operator be an operator for the "concat" operation, given inputs, axis, and options.
-
Set output.
[[operator]]
to operator. -
Set operator’s inputs to inputs.
-
Set operator’s output to output.
-
-
Return output.
7.7.10. conv2d
Compute a 2-D convolution given 4-D input and filter tensorsenum {
MLConv2dFilterOperandLayout ,
"oihw" ,
"hwio" ,
"ohwi" };
"ihwo" dictionary :
MLConv2dOptions MLOperatorOptions {sequence <[EnforceRange ]unsigned long >padding ;sequence <[EnforceRange ]unsigned long >strides ;sequence <[EnforceRange ]unsigned long >dilations ; [EnforceRange ]unsigned long groups = 1;MLInputOperandLayout inputLayout = "nchw";MLConv2dFilterOperandLayout filterLayout = "oihw";MLOperand bias ; };partial interface MLGraphBuilder {MLOperand conv2d (MLOperand input ,MLOperand filter ,optional MLConv2dOptions options = {}); };dictionary {
MLConv2dSupportLimits MLSupportLimits input ;MLSupportLimits filter ;MLSupportLimits bias ;MLSupportLimits output ; };partial dictionary MLOpSupportLimits {MLConv2dSupportLimits conv2d ; };
MLConv2dOptions
has the following members:
padding
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 4: [beginningHeight, endingHeight, beginningWidth, endingWidth]. Specifies the additional rows and columns added to the beginning and ending of each spatial dimension of the convolution input. The default value is [0, 0, 0, 0].
strides
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 2: [strideHeight, strideWidth]. Specifies the stride of the sliding window for each spatial dimension of the convolution input. The default value is [1, 1].
dilations
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 2: [dilationHeight, dilationWidth]. Specifies the dilation factor for each spatial dimension applied on the convolution filter (kernel). The default value is [1, 1].
groups
, of type unsigned long, defaulting to1
-
The number of groups that input channels and output channels are divided into.
inputLayout
, of type MLInputOperandLayout, defaulting to"nchw"
-
Specifies the layout format of the input and output tensor as follows:
filterLayout
, of type MLConv2dFilterOperandLayout, defaulting to"oihw"
-
Specifies the layout format of the filter tensor as follows:
bias
, of type MLOperand-
An additional 1-D tensor with the shape of [outputChannels] whose values are to be added to the convolution result.
-
input
: anMLOperand
. The input 4-D tensor. The logical shape is interpreted according to the value of options.inputLayout
. -
filter
: anMLOperand
. The filter 4-D tensor. The logical shape is interpreted according to the value of options.filterLayout
and options.groups
. -
options
: anMLConv2dOptions
. The optional parameters of the operation.
Returns: an MLOperand
. The output 4-D tensor that contains the convolution result. The output shape is interpreted according to the options.inputLayout
value. More specifically, the spatial dimensions or the sizes of the last two dimensions of the output tensor for the "nchw"
input layout can be calculated as follows:
outputSize = 1 + (inputSize - (filterSize - 1) * dilation - 1 + beginningPadding + endingPadding) / stride
MLConv2dSupportLimits
has following members:
input
, of type MLSupportLimits-
MLSupportLimits
for input operand. filter
, of type MLSupportLimits-
MLSupportLimits
for filter operand. bias
, of type MLSupportLimits-
MLSupportLimits
for bias operand. output
, of type MLSupportLimits-
MLSupportLimits
for output operand.
MLOpSupportLimits
has following member for conv2d()
:
conv2d
, of type MLConv2dSupportLimits-
Support limits for operator
conv2d()
.
"oihw"
layout, [height, width, 1, options.groups] for "hwio"
layout, [options.groups, height, width, 1] for "ohwi"
layout and [1, height, width, options.groups] for "ihwo"
layout. To calculate conv output size given unsigned integers inputSize, filterSize, beginningPadding, endingPadding, stride and dilation, perform these steps. They return a number.
-
Let effectiveFilterSize be ( filterSize - 1 ) * dilation + 1.
-
Let outputSize be ( inputSize - effectiveFilterSize + beginningPadding + endingPadding ) / stride + 1.
-
Return outputSize.
To calculate conv2d output sizes given unsigned integers inputHeight, inputWidth, filterHeight and filterWidth, list of 4 unsigned integers padding, list of 2 unsigned integers strides, and list of 2 unsigned integers dilations, perform these steps. They return a list of 2 numbers.
-
Let outputHeight be the result of calculating conv output size given inputHeight, filterHeight, padding[0], padding[1], strides[0] and dilations[0].
-
Let outputWidth be the result of calculating conv output size given inputWidth, filterWidth, padding[2], padding[3], strides[1] and dilations[1].
-
Return « outputHeight, outputWidth ».
The conv2d(input, filter, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of input, filter, and options.
bias
(if it exists) returns false, then throw aTypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
If filter’s dataType is not equal to input’s dataType, then throw a
TypeError
. -
If options.
padding
does not exist, set it to the list « 0, 0, 0, 0 ». -
Otherwise, if options.
padding
's size is not 4, then throw aTypeError
. -
If options.
strides
does not exist, set it to the list « 1, 1 ». -
Otherwise, if options.
strides
's size is not 2, then throw aTypeError
. -
If any element in options.
strides
is equal to 0, then throw aTypeError
. -
If options.
dilations
does not exist, set it to the list « 1, 1 ». -
Otherwise, if options.
dilations
's size is not 2, then throw aTypeError
. -
If any element in options.
dilations
is equal to 0, then throw aTypeError
. -
Calculate the output shape:
-
Let inputShape be input’s shape.
-
Switch on options.
inputLayout
: -
Let filterShape be filter’s shape.
-
Switch on options.
filterLayout
:"hwio"
-
-
Let filterHeight be filterShape[0].
-
Let filterWidth be filterShape[1].
-
Let filterInputChannels be filterShape[2].
-
Let outputChannels be filterShape[3].
-
"ohwi"
-
-
Let outputChannels be filterShape[0].
-
Let filterHeight be filterShape[1].
-
Let filterWidth be filterShape[2].
-
Let filterInputChannels be filterShape[3].
-
"ihwo"
-
-
Let filterInputChannels be filterShape[0].
-
Let filterHeight be filterShape[1].
-
Let filterWidth be filterShape[2].
-
Let outputChannels be filterShape[3].
-
"oihw"
-
-
Let outputChannels be filterShape[0].
-
Let filterInputChannels be filterShape[1].
-
Let filterHeight be filterShape[2].
-
Let filterWidth be filterShape[3].
-
-
If inputChannels % options.
groups
is not 0, then throw aTypeError
. -
Otherwise, if inputChannels / options.
groups
is not equal to filterInputChannels, then throw aTypeError
. -
Let outputSizes be the result of calculating conv2d output sizes given inputHeight, inputWidth, filterHeight, filterWidth, options.
padding
, options.strides
, and options.dilations
. -
Switch on options.
inputLayout
: -
If any item in outputShape is not a valid dimension, then throw a
TypeError
. -
Let desc be the result of creating an MLOperandDescriptor given input’s dataType and outputShape.
-
-
Make graph connections:
-
Let output be the result of creating an MLOperand given this and desc.
-
Let operator be an operator for the "conv2d" operation, given options and filter.
-
Set output.
[[operator]]
to operator. -
Set operator’s inputs to input and filter.
-
Set operator’s output to output.
-
-
Return output.
7.7.11. convTranspose2d
Compute a 2-D transposed convolution given 4-D input and filter tensorsenum {
MLConvTranspose2dFilterOperandLayout ,
"iohw" ,
"hwoi" };
"ohwi" dictionary :
MLConvTranspose2dOptions MLOperatorOptions {sequence <[EnforceRange ]unsigned long >padding ;sequence <[EnforceRange ]unsigned long >strides ;sequence <[EnforceRange ]unsigned long >dilations ;sequence <[EnforceRange ]unsigned long >outputPadding ;sequence <[EnforceRange ]unsigned long >outputSizes ; [EnforceRange ]unsigned long groups = 1;MLInputOperandLayout inputLayout = "nchw";MLConvTranspose2dFilterOperandLayout filterLayout = "iohw";MLOperand bias ; };partial interface MLGraphBuilder {MLOperand convTranspose2d (MLOperand input ,MLOperand filter ,optional MLConvTranspose2dOptions options = {}); };partial dictionary MLOpSupportLimits {MLConv2dSupportLimits convTranspose2d ; };
MLConvTranspose2dOptions
has the following members:
padding
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 4: [beginningHeight, endingHeight, beginningWidth, endingWidth]. Specifies the additional rows and columns added to the beginning and ending of each spatial dimension of the convolution input. The default value is [0, 0, 0, 0].
strides
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 2: [strideHeight, strideWidth]. Specifies the stride of the sliding window for each spatial dimension of the convolution input. The default value is [1, 1].
dilations
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 2: [dilationHeight, dilationWidth]. Specifies the dilation factor for each spatial dimension applied on the convolution filter (kernel). The default value is [1, 1].
outputPadding
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 2. Specifies the padding values applied to each spatial dimension of the output tensor. The explicit padding values are needed to disambiguate the output tensor shape for transposed convolution when the value of the options.
strides
is greater than 1.Note that these values are only used to disambiguate output shape when needed; it does not necessarily cause any padding value to be written to the output tensor.
The default value is [0, 0].
outputSizes
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 2. Specifies the sizes of the last two dimensions of the output tensor. When the output sizes are explicitly specified, the output padding values in
outputPadding
are ignored.If not specified, the output sizes are automatically computed.
groups
, of type unsigned long, defaulting to1
-
The number of groups that input channels and output channels are divided into.
inputLayout
, of type MLInputOperandLayout, defaulting to"nchw"
-
Specifies the layout format of the input and output tensor as follows:
filterLayout
, of type MLConvTranspose2dFilterOperandLayout, defaulting to"iohw"
-
Specifies the layout format of the filter tensor as follows:
bias
, of type MLOperand-
An additional 1-D tensor with the shape of [outputChannels] whose values are to be added to the convolution result.
-
input
: anMLOperand
. The input 4-D tensor. The logical shape is interpreted according to the value of options.inputLayout
. -
filter
: anMLOperand
. The filter 4-D tensor. The logical shape is interpreted according to the value of options.filterLayout
andgroups
. -
options
: an optionalMLConvTranspose2dOptions
.
Returns: an MLOperand
. The output 4-D tensor that contains the transposed convolution result. The output shape is interpreted according to the options.inputLayout
value. More specifically, unless the options.outputSizes
values are explicitly specified, the options.outputPadding
may be needed to compute the spatial dimension values of the output tensor as follows:
outputSize = (inputSize - 1) * stride + (filterSize - 1) * dilation + 1 - beginningPadding - endingPadding + outputPadding
MLOpSupportLimits
has following member for :
convTranspose2d
, of type MLConv2dSupportLimits-
Support limits for operator
convTranspose2d()
.
To calculate convtranspose output size given unsigned integers inputSize, filterSize, beginningPadding, endingPadding, stride, dilation, and outputPadding, perform these steps. They return a number.
-
Let effectiveFilterSize be ( filterSize - 1 ) * dilation + 1.
-
Let outputSize be ( inputSize - 1 ) * stride + effectiveFilterSize - beginningPadding - endingPadding + outputPadding.
-
Return outputSize.
To calculate convtranspose2d output sizes given unsigned integers inputHeight, inputWidth, filterHeight and filterWidth, list of 4 unsigned integers padding, list of 2 unsigned integers strides, list of 2 unsigned integers dilations, and list of 2 unsigned integers outputPadding, perform these steps. They return a list of 2 numbers.
-
Let outputHeight be the result of calculating convtranspose output size given inputHeight, filterHeight, padding[0], padding[1], strides[0], dilations[0], and outputPadding[0].
-
Let outputWidth be the result of calculating convtranspose output size given inputWidth, filterWidth, padding[2], padding[3], strides[1], dilations[1] and outputPadding[1].
-
Return « outputHeight, outputWidth ».
The convTranspose2d(input, filter, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of input, filter, and options.
bias
(if it exists) returns false, then throw aTypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
If filter’s dataType is not equal to input’s dataType, then throw a
TypeError
. -
If options.
padding
does not exist, set it to the list « 0, 0, 0, 0 ». -
Otherwise, if options.
padding
's size is not 4, then throw aTypeError
. -
If options.
strides
does not exist, set it to the list « 1, 1 ». -
Otherwise, if options.
strides
's size is not 2, then throw aTypeError
. -
If any element in options.
strides
is equal to 0, then throw aTypeError
. -
If options.
dilations
does not exist, set it to the list « 1, 1 ». -
Otherwise, if options.
dilations
's size is not 2, then throw aTypeError
. -
If any element in options.
dilations
is equal to 0, then throw aTypeError
. -
If options.
outputPadding
does not exist, set it to the list « 0, 0 ». -
Otherwise, if options.
outputPadding
's size is not 2, then throw aTypeError
. -
If options.
outputSizes
exists: -
Otherwise:
-
If options.
outputPadding
[0] is greater than or equal to options.strides
[0], or options.outputPadding
[1] is greater than or equal to options.strides
[1], then throw aTypeError
.
-
-
Calculate the output shape:
-
Let inputShape be input’s shape.
-
Switch on options.
inputLayout
: -
Let filterShape be filter’s shape.
-
Switch on options.
filterLayout
:"iohw"
-
-
Let filterInputChannels be filterShape[0].
-
Let filterOutputChannels be |filterShape[1].
-
Let filterHeight be filterShape[2].
-
Let filterWidth be filterShape[3].
-
"hwoi"
-
-
Let filterHeight be filterShape[0].
-
Let filterWidth be filterShape[1].
-
Let filterOutputChannels be |filterShape[2].
-
Let filterInputChannels be filterShape[3].
-
"ohwi"
-
-
Let filterOutputChannels be |filterShape[0].
-
Let filterHeight be filterShape[1].
-
Let filterWidth be filterShape[2].
-
Let filterInputChannels be filterShape[3].
-
-
If inputChannels is not equal to filterInputChannels, then throw a
TypeError
. -
Let outputChannels be filterOutputChannels * options.
groups
. -
If options.
outputSizes
exists, let outputSizes be options.outputSizes
. -
Otherwise, let outputSizes be the result of calculating convtranspose2d output sizes given inputHeight, inputWidth, filterHeight, filterWidth, options.
padding
, options.strides
, options.dilations
, and options.outputPadding
. -
Switch on options.
inputLayout
: -
If any item in outputShape is not a valid dimension, then throw a
TypeError
. -
Let desc be the result of creating an MLOperandDescriptor given input’s dataType and outputShape.
-
-
Make graph connections:
-
Let output be the result of creating an MLOperand given this and desc.
-
Let operator be an operator for the "convTranspose2d" operation, given options and filter.
-
Set output.
[[operator]]
to operator. -
Set operator’s inputs to input and filter.
-
Set operator’s output to output.
-
-
Return output.
7.7.12. Element-wise binary operations
Compute the element-wise binary addition, subtraction, multiplication, division, power, maximum and minimum of the two input tensors.The operation will be broadcast according to [numpy-broadcasting-rule]. The input tensors must be bidirectionally broadcastable. The rank of the output tensor is the maximum rank of the input tensors. For each dimension of the output tensor, its size is the maximum size along that dimension of the input tensors.
partial interface MLGraphBuilder {MLOperand add (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {});MLOperand sub (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {});MLOperand mul (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {});MLOperand div (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {});MLOperand max (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {});MLOperand min (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {});MLOperand pow (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {}); };partial dictionary MLOpSupportLimits {MLBinarySupportLimits add ;MLBinarySupportLimits sub ;MLBinarySupportLimits mul ;MLBinarySupportLimits div ;MLBinarySupportLimits max ;MLBinarySupportLimits min ;MLBinarySupportLimits pow ; };
-
a
: anMLOperand
. The first input tensor. -
b
: anMLOperand
. The second input tensor. -
options
: anMLOperatorOptions
. Specifies the optional parameters of the operation.
Returns: an MLOperand
. The output tensor that contains the result of
element-wise binary operation of the two input tensors.
-
add: Add the values of the two input tensors, element-wise.
-
sub: Subtract the values of the second input tensor from the values of the first input tensor, element-wise.
-
mul: Multiply the values of the two input tensors, element-wise.
-
div: Divide the values of the first input tensor with the values of the second tensor, element-wise.
-
max: Select the greater values of the two input tensors, element-wise.
-
min: Select the lesser values of the two input tensors, element-wise.
-
pow: Compute the values of the values of the first input tensor to the power of the values of the second input tensor, element-wise.
MLOpSupportLimits
has following members for eleemntwise-binary operations:
add
, of type MLBinarySupportLimits-
Support limits for operator
add()
. sub
, of type MLBinarySupportLimits-
Support limits for operator
sub()
. mul
, of type MLBinarySupportLimits-
Support limits for operator
mul()
. div
, of type MLBinarySupportLimits-
Support limits for operator
div()
. max
, of type MLBinarySupportLimits-
Support limits for operator
max()
. min
, of type MLBinarySupportLimits-
Support limits for operator
min()
. pow
, of type MLBinarySupportLimits-
Support limits for operator
pow()
.
To create element-wise binary operation given string op, MLOperand
a, MLOperand
b, and MLOperatorOptions
options, run the following steps:
-
Assert: op is one of "add", "sub", "mul", "div", "max", "min", "pow".
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of a and b returns false, then throw a
TypeError
. -
If a’s dataType is not equal to b’s dataType, then throw a
TypeError
. -
Let outputShape be the result of bidirectionally broadcasting a’s shape and b’s shape.
-
Let descriptor be the result of creating an MLOperandDescriptor given a’s dataType and outputShape.
-
Make graph connections:
-
Let output be the result of creating an MLOperand given this and descriptor.
-
Let operator be an operator for the op operation, given a, b, and options.
-
Set output.
[[operator]]
to operator. -
Set operator’s inputs to a and b.
-
Set operator’s output to output.
-
-
Return output.
The element-wise binary operation algorithms invoke the create element-wise binary operation steps as follows.
add(a, b, options)
method steps are:
-
Let output be the result of running the create element-wise binary operation given "add", a, b, and options.
-
Return output.
sub(a, b, options)
method steps are:
-
Let output be the result of running the create element-wise binary operation given "sub", a, b, and options.
-
Return output.
mul(a, b, options)
method steps are:
-
Let output be the result of running the create element-wise binary operation given "mul", a, b, and options.
-
Return output.
div(a, b, options)
method steps are:
-
Let output be the result of running the create element-wise binary operation given "div", a, b, and options.
-
Return output.
max(a, b, options)
method steps are:
-
Let output be the result of running the create element-wise binary operation given "max", a, b, and options.
-
Return output.
min(a, b, options)
method steps are:
-
Let output be the result of running the create element-wise binary operation given "min", a, b, and options.
-
Return output.
pow(a, b, options)
method steps are:
-
Let output be the result of running the create element-wise binary operation given "pow", a, b, and options.
-
Return output.
7.7.13. Element-wise logical operations
Compare input tensors element-wise and return a"uint8"
tensor of values 0 (false) or 1 (true) for the comparisons. For single-operand operations, return the logical results of the operation.
For multiple-operand operations, the operation will be broadcast according to [numpy-broadcasting-rule]. The input tensors must be bidirectionally broadcastable. The rank of the output tensor is the maximum rank of the input tensors. For each dimension of the output tensor, its size is the maximum size along that dimension of the input tensors.
partial interface MLGraphBuilder {MLOperand equal (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {});MLOperand greater (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {});MLOperand greaterOrEqual (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {});MLOperand lesser (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {});MLOperand lesserOrEqual (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {});MLOperand logicalNot (MLOperand a ,optional MLOperatorOptions options = {}); };dictionary {
MLLogicalNotSupportLimits MLSupportLimits a ;MLSupportLimits output ; };partial dictionary MLOpSupportLimits {MLBinarySupportLimits equal ;MLBinarySupportLimits greater ;MLBinarySupportLimits greaterOrEqual ;MLBinarySupportLimits lesser ;MLBinarySupportLimits lesserOrEqual ;MLLogicalNotSupportLimits logicalNot ; };
-
a
: anMLOperand
. The first input tensor. -
b
: anMLOperand
. The second input tensor when specified. -
options
: anMLOperatorOptions
. Specifies the optional parameters of the operation.
Returns: an MLOperand
. The output tensor that contains the result of element-wise comparison of the two input tensors.
MLLogicalNotSupportLimits
has following members:
a
, of type MLSupportLimits-
MLSupportLimits
for a operand. output
, of type MLSupportLimits-
MLSupportLimits
for output operand.
MLOpSupportLimits
has following members for element-wise logical operations:
equal
, of type MLBinarySupportLimits-
Support limits for operator
equal()
. greater
, of type MLBinarySupportLimits-
Support limits for operator
greater()
. greaterOrEqual
, of type MLBinarySupportLimits-
Support limits for operator
greaterOrEqual()
. lesser
, of type MLBinarySupportLimits-
Support limits for operator
lesser()
. lesserOrEqual
, of type MLBinarySupportLimits-
Support limits for operator
lesserOrEqual()
. logicalNot
, of type MLLogicalNotSupportLimits-
Support limits for operator
logicalNot()
.
-
equal: Compare if the values of the two input tensors are equal, element-wise.
-
greater: Compare if the values of the first input tensor is greater, element-wise.
-
greaterOrEqual: Compare if the values of the first input tensor is greater or equal, element-wise.
-
lesser: Compare if the values of the first input tensor is lesser, element-wise.
-
lesserOrEqual: Compare if the values of the first input tensor is lesser or equal, element-wise.
-
logicalNot: Invert the values of the input tensor to values 0 or 1, element-wise. Specifically, when the input value is non-zero, invert it to 0. Conversely, for a zero input value, invert it to 1.
greaterOrEqual()
and lesserOrEqual()
can each be implemented in terms of operations logicalNot()
, lesser()
, and greater()
in other words builder.greaterOrEqual(a, b)
is builder.logicalNot(builder.lesser(a, b))
, they are specifically defined to handle NaN cases and for performance reason to avoid double comparisons. To create element-wise logical operation given string op, MLOperand
a, an optional MLOperand
b, and MLOperatorOptions
options, run the following steps:
-
Assert: op is one of "equal", "greater", "greaterOrEqual", "lesser", "lesserOrEqual", "logicalNot".
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If op is "logicalNot":
-
Otherwise:
-
Let descriptor be the result of creating an MLOperandDescriptor given
"uint8"
and outputShape. -
Make graph connections:
-
Let output be the result of creating an MLOperand given this and descriptor.
-
Let operator be an operator for the op operation, given a and (if op is not "logicalNot") b, and options.
-
Set output.
[[operator]]
to operator. -
Set operator’s inputs to a and (if op is anything other than "logicalNot") b.
-
Set operator’s output to output.
-
-
Return output.
The element-wise logical operation algorithms invoke the create element-wise logical operation steps as follows.
equal(a, b, options)
method steps are:
-
Let output be the result of running the create element-wise logical operation given "equal", a, b, and options.
-
Return output.
greater(a, b, options)
method steps are:
-
Let output be the result of running the create element-wise logical operation given "greater", a, b, and options.
-
Return output.
greaterOrEqual(a, b, options)
method steps are:
-
Let output be the result of running the create element-wise logical operation given "greaterOrEqual", a, b, and options.
-
Return output.
lesser(a, b, options)
method steps are:
-
Let output be the result of running the create element-wise logical operation given "lesser", a, b, and options.
-
Return output.
lesserOrEqual(a, b, options)
method steps are:
-
Let output be the result of running the create element-wise logical operation given "lesserOrEqual", a, b, and options.
-
Return output.
logicalNot(a, options)
method steps are:
-
Let output be the result of running the create element-wise logical operation given "logicalNot", a, and options.
-
Return output.
7.7.14. Element-wise unary operations
Compute the element-wise unary operation for input tensor.partial interface MLGraphBuilder {MLOperand abs (MLOperand input ,optional MLOperatorOptions options = {});MLOperand ceil (MLOperand input ,optional MLOperatorOptions options = {});MLOperand cos (MLOperand input ,optional MLOperatorOptions options = {});MLOperand erf (MLOperand input ,optional MLOperatorOptions options = {});MLOperand exp (MLOperand input ,optional MLOperatorOptions options = {});MLOperand floor (MLOperand input ,optional MLOperatorOptions options = {});MLOperand identity (MLOperand input ,optional MLOperatorOptions options = {});MLOperand log (MLOperand input ,optional MLOperatorOptions options = {});MLOperand neg (MLOperand input ,optional MLOperatorOptions options = {});MLOperand reciprocal (MLOperand input ,optional MLOperatorOptions options = {});MLOperand sin (MLOperand input ,optional MLOperatorOptions options = {});MLOperand sqrt (MLOperand input ,optional MLOperatorOptions options = {});MLOperand tan (MLOperand input ,optional MLOperatorOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits abs ;MLSingleInputSupportLimits ceil ;MLSingleInputSupportLimits cos ;MLSingleInputSupportLimits erf ;MLSingleInputSupportLimits exp ;MLSingleInputSupportLimits floor ;MLSingleInputSupportLimits identity ;MLSingleInputSupportLimits log ;MLSingleInputSupportLimits neg ;MLSingleInputSupportLimits reciprocal ;MLSingleInputSupportLimits sin ;MLSingleInputSupportLimits sqrt ;MLSingleInputSupportLimits tan ; };
-
input
: anMLOperand
. The input tensor. -
options
: anMLOperatorOptions
. Specifies the optional parameters of the operation.
Returns: an MLOperand
. The output tensor that contains the result of
element-wise unary operation of the input tensor. The shape of the output
tensor is the same as the shape of input tensor.
MLOpSupportLimits
has following members for element-wise unary operations:
abs
, of type MLSingleInputSupportLimits-
Support limits for operator
abs()
. ceil
, of type MLSingleInputSupportLimits-
Support limits for operator
ceil()
. cos
, of type MLSingleInputSupportLimits-
Support limits for operator
cos()
. erf
, of type MLSingleInputSupportLimits-
Support limits for operator
erf()
. exp
, of type MLSingleInputSupportLimits-
Support limits for operator
exp()
. floor
, of type MLSingleInputSupportLimits-
Support limits for operator
floor()
. identity
, of type MLSingleInputSupportLimits-
Support limits for operator
identity()
. log
, of type MLSingleInputSupportLimits-
Support limits for operator
log()
. neg
, of type MLSingleInputSupportLimits-
Support limits for operator
neg()
. reciprocal
, of type MLSingleInputSupportLimits-
Support limits for operator
reciprocal()
. sin
, of type MLSingleInputSupportLimits-
Support limits for operator
sin()
. sqrt
, of type MLSingleInputSupportLimits-
Support limits for operator
sqrt()
. tan
, of type MLSingleInputSupportLimits-
Support limits for operator
tan()
.
-
abs: Compute the absolute value of the input tensor, element-wise.
-
ceil: Compute the ceiling of the input tensor, element-wise.
-
cos: Compute the cosine of the input tensor, element-wise.
-
erf: Compute the error function [Error-Function] of the input tensor, element-wise.
-
exp: Compute the exponential of the input tensor, element-wise.
-
floor: Compute the floor of the input tensor, element-wise.
-
identity: Copy the value of the input tensor to the output tensor, element-wise.
-
log: Compute the natural logarithm of the input tensor, element-wise.
-
neg: Compute the numerical negative value of the input tensor, element-wise.
-
reciprocal: Compute the reciprocal of the input tensor, element-wise.
-
sin: Compute the sine of the input tensor, element-wise.
-
sqrt: Compute the square root of the input tensor, element-wise.
-
tan: Compute the tangent of the input tensor, element-wise.
To create element-wise unary operation given string op, MLOperand
input, optional list allowedDataTypes, and options, run the following steps:
-
Assert: op is one of "abs", "ceil", "cos", "erf", "exp", "floor", "identity", "log", "neg", "reciprocal", "sin", "sqrt", "tan".
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
If allowedDataTypes is given and it does not contain input’s dataType, then throw a
TypeError
. -
Make graph connections:
-
Let output be the result of copying an MLOperand given input.
-
Let operator be an operator for the op operation given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
The element-wise unary operation algorithms invoke the create element-wise unary operation steps as follows.
abs(input, options)
method steps are:
ceil(input, options)
method steps are:
-
Let output be the result of running the create element-wise unary operation given "ceil", input, «
"float32"
,"float16"
», and options. -
Return output.
cos(input, options)
method steps are:
-
Let output be the result of running the create element-wise unary operation given "cos", input, «
"float32"
,"float16"
», and options. -
Return output.
erf(input, options)
method steps are:
-
Let output be the result of running the create element-wise unary operation given "erf", input, «
"float32"
,"float16"
», and options. -
Return output.
exp(input, options)
method steps are:
-
Let output be the result of running the create element-wise unary operation given "exp", input, «
"float32"
,"float16"
», and options. -
Return output.
floor(input, options)
method steps are:
-
Let output be the result of running the create element-wise unary operation given "floor", input, «
"float32"
,"float16"
», and options. -
Return output.
identity(input, options)
method steps are:
-
Let output be the result of running the create element-wise unary operation given "identity" input, and options.
-
Return output.
log(input, options)
method steps are:
-
Let output be the result of running the create element-wise unary operation given "log", input, «
"float32"
,"float16"
», and options. -
Return output.
neg(input, options)
method steps are:
reciprocal(input, options)
method steps are:
-
Let output be the result of running the create element-wise unary operation given "reciprocal", input, «
"float32"
,"float16"
», and options. -
Return output.
sin(input, options)
method steps are:
-
Let output be the result of running the create element-wise unary operation given "sin", input, «
"float32"
,"float16"
», and options. -
Return output.
sqrt(input, options)
method steps are:
-
Let output be the result of running the create element-wise unary operation given "sqrt", input, «
"float32"
,"float16"
», and options. -
Return output.
tan(input, options)
method steps are:
-
Let output be the result of running the create element-wise unary operation given "tan", input, «
"float32"
,"float16"
», and options. -
Return output.
7.7.15. elu
Calculate the exponential linear unit function (ELU) on the input tensor element-wise. The calculation follows the expressionmax(0, x) + alpha * (exp(min(0, x)) - 1)
.
dictionary :
MLEluOptions MLOperatorOptions {double alpha = 1; };partial interface MLGraphBuilder {MLOperand elu (MLOperand input ,optional MLEluOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits elu ; };
MLEluOptions
has the following members:
alpha
, of type double, defaulting to1
-
A scalar multiplier.
-
input
: anMLOperand
. The input tensor. -
options
: an optionalMLEluOptions
. The optional parameters of the operation.
Returns:
-
an
MLOperand
. The output tensor of the same shape as input.
MLOpSupportLimits
has following members for elu()
:
elu
, of type MLSingleInputSupportLimits-
Support limits for operator
elu()
.
The elu(input, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
Set options.
alpha
to the result of casting options.alpha
to input’s dataType. -
Make graph connections:
-
Let output be the result of copying an MLOperand given input.
-
Let operator be an operator for the "elu" operation, given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function elu( builder, input, options) { return builder. add( builder. max( builder. constant( input. dataType(), 0 ), input), builder. mul( builder. constant( input. dataType(), options. alpha), builder. sub( builder. exp( builder. min( builder. constant( input. dataType(), 0 ), input)), builder. constant( input. dataType(), 1 )))); }
7.7.16. expand
Expand any dimension of size 1 of the input tensor to a larger size according to the new shape. The expansion is consistent with [numpy-broadcasting-rule]. The input tensor must be unidirectionally broadcastable to the new shape; each dimension must be of size 1 or match the sizes of the corresponding output dimensions according to the new shape.partial interface MLGraphBuilder {MLOperand expand (MLOperand input ,sequence <[EnforceRange ]unsigned long >newShape ,optional MLOperatorOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits expand ; };
-
input
: anMLOperand
. An input tensor -
newShape
: sequence<unsigned long
>. The new shape the input tensor is expanded to. -
options
: anMLOperatorOptions
. Specifies the optional parameters of the operation.
Returns: an MLOperand
. The tensor with expanded size shape.
MLOpSupportLimits
has following members for expand()
:
expand
, of type MLSingleInputSupportLimits-
Support limits for operator
expand()
.
The expand(input, newShape, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
Let outputShape be the result of unidirectionally broadcasting input’s shape and newShape.
-
Let outputDescriptor be the result of creating an MLOperandDescriptor given input’s dataType and outputShape.
-
Make graph connections:
-
Let output be the result of creating an MLOperand given this and outputDescriptor.
-
Let operator be an operator for the "expand" operation, given input, newShape, and options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
7.7.17. gather
Gather values of the input tensor along an axis according to the indices.dictionary :
MLGatherOptions MLOperatorOptions { [EnforceRange ]unsigned long axis = 0; };partial interface MLGraphBuilder {MLOperand gather (MLOperand input ,MLOperand indices ,optional MLGatherOptions options = {}); };dictionary {
MLGatherSupportLimits MLSupportLimits input ;MLSupportLimits indices ;MLSupportLimits output ; };partial dictionary MLOpSupportLimits {MLGatherSupportLimits gather ; };
MLGatherOptions
has the following members:
axis
, of type unsigned long, defaulting to0
-
The axis along which the gathered values are obtained. Its value must be in the range [0, N-1] where N is the rank of the input tensor.
-
input
: anMLOperand
. The input N-D tensor from which the values are gathered. -
indices
: anMLOperand
. The indices N-D tensor of the input values to gather. The values must be of type"int32"
,"uint32"
or"int64"
, and must be in the range -N (inclusive) to N (exclusive) where N is the size of the input dimension indexed by options.axis, and a negative index means indexing from the end of the dimension. -
options
: an optionalMLGatherOptions
. The optional parameters of the operation.
Returns: an MLOperand
. The output N-D tensor of rank equal to the rank of input + the rank of indices - 1.
MLGatherSupportLimits
has following members:
input
, of type MLSupportLimits-
MLSupportLimits
for input operand. indices
, of type MLSupportLimits-
MLSupportLimits
for indices operand. output
, of type MLSupportLimits-
MLSupportLimits
for output operand.
MLOpSupportLimits
has following members for gather()
:
gather
, of type MLGatherSupportLimits-
Support limits for operator
gather()
.
indices
parameter to gather()
can not be clamped to the allowed range when the graph is built because the inputs are not known until execution. Implementations can introduce clamp()
in the compiled graph if the required clamping behavior is not provided by the underlying platform. Similarly, if the underlying platform does not support negative indices, the implementation can introduce operations in the compiled graph to transform a negative index from the end of the dimension into a positive index. The gather(input, indices, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of input and indices returns false, then throw a
TypeError
. -
If indices’s dataType is not
"int32"
,"uint32"
or"int64"
, then throw aTypeError
. -
Let shapeInput be input’s shape and rankInput be shapeInput’s rank.
-
Let shapeIndices be indices’s shape.
-
Let axis be options.
axis
. -
If axis is greater than or equal to rankInput, then throw a
TypeError
. -
Let dimCount be zero.
-
Let rankOutput be zero.
-
Let shapeOutput be an empty list.
-
For each size of shapeInput:
-
If dimCount is equal to axis then break.
-
Set shapeOutput[dimCount] to size.
-
Increment dimCount by one.
-
-
Set rankOutput to dimCount.
-
Let dimCount be zero.
-
For each size of shapeIndices:
-
Set shapeOutput[rankOutput + dimCount] to size.
-
Increment dimCount by one.
-
-
Set rankOutput to rankOutput + dimCount.
-
Let dimCount be zero.
-
For each size of shapeInput:
-
If dimCount is less than or equal to axis then continue.
-
Set shapeOutput[rankOutput + dimCount - axis - 1] to size.
-
Increment dimCount by one.
-
-
Let desc be the result of creating an MLOperandDescriptor given input’s dataType and shapeOutput.
-
Make graph connections:
-
Let output be the result of creating an MLOperand given desc.
-
Let operator be an operator for the "gather" operation, given input, indices, and options.
-
Set output.
[[operator]]
to operator. -
Set operator’s inputs to input and indices.
-
Set operator’s output to output.
-
-
Return output.
Examples of how gather works in different slicing schemes.
// input of shape [4,3]: // [[ 0, 1, 2], // [10, 11, 12], // [20, 21, 22], // [30, 31, 32]] const input= builder. constant( { shape: [ 4 , 3 ]}, new Float32Array([ 0 , 1 , 2 , 10 , 11 , 12 , 20 , 21 , 22 , 30 , 31 , 32 ])); const indices1= builder. constant( { dataType: 'uint32' , shape: [ 2 ]}, new Uint32Array([ 3 , 1 ])); const indices2= builder. constant( { dataType: 'uint32' , shape: [ 3 ]}, new Uint32Array([ 2 , 1 , 1 ])); const indices3= builder. constant( { dataType: 'uint32' , shape: [ 2 , 2 ]}, new Uint32Array([ 0 , 1 , 1 , 2 ])); // axis = 0 (default) // indices of shape [2]: // [3,1] // output of shape [2,3]: // [[30, 31, 32], // [10, 11, 12]] const output1= builder. gather( input, indices1); // axis = 1 // indices of shape [3]: // [2,1,1] // output of shape [4,3]: // [[ 2, 1, 1], // [12, 11, 11], // [22, 21, 21], // [32, 31, 31]] const output2= builder. gather( input, indices2, { axis: 1 }); // axis = 1 // indices of shape [2,2]: // [[0, 1], // [1, 2]] // output of shape [4,2,2]: // [[[ 0, 1], [ 1, 2]], // [[10, 11], [11, 12]], // [[20, 21], [21, 22]], // [[30, 31], [31, 32]]] const output3= builder. gather( input, indices3, { axis: 1 });
7.7.18. gelu
Compute the gaussian error linear unit function (GELU) of the input tensor. The calculation follows the expression0.5 * x * (1 + erf(x / sqrt(2)))
.
partial interface MLGraphBuilder {MLOperand gelu (MLOperand input ,optional MLOperatorOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits gelu ; };
-
input
: anMLOperand
. The input tensor. -
options
: anMLOperatorOptions
. Specifies the optional parameters of the operation.
Returns:
-
an
MLOperand
. The output tensor of the same shape as input.
MLOpSupportLimits
has following member for gelu()
:
gelu
, of type MLSingleInputSupportLimits-
Support limits for operator
gelu()
.
The gelu(input, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
Make graph connections:
-
Let output be the result of copying an MLOperand given input.
-
Let operator be an operator for the "gelu" operation given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function gelu( builder, input) { return builder. mul( builder. mul( input, builder. constant( input. dataType(), 0.5 )), builder. add( builder. constant( input. dataType(), 1 ), builder. erf( builder. div( input, builder. sqrt( builder. constant( input. dataType(), 2 )))))); }
7.7.19. gemm
Calculate the general matrix multiplication of the Basic Linear Algebra Subprograms. The calculation follows the expressionalpha * A * B + beta * C
, where A
is a 2-D tensor with shape [M, K] or [K, M], B
is a 2-D tensor with shape [K, N] or [N, K], and C
is unidirectionally broadcastable to the shape [M, N]. A
and B
may optionally be transposed prior to the calculation.
dictionary :
MLGemmOptions MLOperatorOptions {MLOperand c ;double alpha = 1.0;double beta = 1.0;boolean aTranspose =false ;boolean bTranspose =false ; };partial interface MLGraphBuilder {MLOperand gemm (MLOperand a ,MLOperand b ,optional MLGemmOptions options = {}); };dictionary {
MLGemmSupportLimits MLSupportLimits a ;MLSupportLimits b ;MLSupportLimits c ;MLSupportLimits output ; };partial dictionary MLOpSupportLimits {MLGemmSupportLimits gemm ; };
MLGemmOptions
has the following members:
c
, of type MLOperand-
The third input tensor. It is either a scalar, or of the shape that is unidirectionally broadcastable to the shape [M, N]. When it is not specified, the computation is done as if c is a scalar 0.0.
alpha
, of type double, defaulting to1.0
-
A multiplier for the first input.
beta
, of type double, defaulting to1.0
-
A multiplier for the third input
c
. aTranspose
, of type boolean, defaulting tofalse
-
Indicates if the first input should be transposed prior to calculating the output.
bTranspose
, of type boolean, defaulting tofalse
-
Indicates if the second input should be transposed prior to calculating the output.
-
a
: anMLOperand
. The first input 2-D tensor with shape [M, K] if aTranspose is false, or [K, M] if aTranspose is true. -
b
: anMLOperand
. The second input 2-D tensor with shape [K, N] if bTranspose is false, or [N, K] if bTranspose is true. -
options
: an optionalMLGemmOptions
. The optional parameters of the operation.
Returns: an MLOperand
. The output 2-D tensor of shape [M, N] that contains the calculated product of all the inputs.
MLGemmSupportLimits
has following members:
a
, of type MLSupportLimits-
MLSupportLimits
for a operand. b
, of type MLSupportLimits-
MLSupportLimits
for b operand. c
, of type MLSupportLimits-
MLSupportLimits
for c operand. output
, of type MLSupportLimits-
MLSupportLimits
for output operand.
MLOpSupportLimits
has following member for gemm()
:
gemm
, of type MLGemmSupportLimits-
Support limits for operator
gemm()
.
The gemm(a, b, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of a and b returns false, then throw a
TypeError
. -
If a’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
If b’s dataType is not equal to a’s dataType, then throw a
TypeError
. -
If a’s rank is not 2 or b’s rank is not 2, then throw a
TypeError
. -
Set options.
alpha
to the result of casting options.alpha
to a’s dataType. -
Set options.
beta
to the result of casting options.beta
to a’s dataType. -
If options.
aTranspose
is true, then reverse the order of the items in shapeA. -
If options.
bTranspose
is true, then reverse the order of the items in shapeB. -
If shapeA[1] is not equal to shapeB[0], then throw a
TypeError
. -
Let desc be the result of creating an MLOperandDescriptor given a’s dataType and « shapeA[0], shapeB[1] ».
-
Make graph connections:
-
Let output be the result of creating an MLOperand given this and desc.
-
Let operator be an operator for the "gemm" operation, given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s inputs to a and b.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function gemm( builder, a, b, options) { if ( options. aTranspose) a= builder. transpose( a); if ( options. bTranspose) b= builder. transpose( b); let ab= builder. matmul( builder. mul( builder. constant( a. dataType(), options. alpha), a), b); return ( options. c? builder. add( ab, builder. mul( builder. constant( a. dataType(), options. beta), options. c)) : ab); }
7.7.20. gru
Gated Recurrent Unit [GRU] recurrent network uses an update, reset, and new gate to compute the output state that rolls into the output across the temporal sequence of the network.enum {
MLGruWeightLayout , // update-reset-new gate ordering
"zrn" // reset-update-new gate ordering };
"rzn" enum {
MLRecurrentNetworkActivation ,
"relu" ,
"sigmoid" };
"tanh" enum {
MLRecurrentNetworkDirection ,
"forward" ,
"backward" };
"both" dictionary :
MLGruOptions MLOperatorOptions {MLOperand bias ;MLOperand recurrentBias ;MLOperand initialHiddenState ;boolean resetAfter =true ;boolean returnSequence =false ;MLRecurrentNetworkDirection direction = "forward";MLGruWeightLayout layout = "zrn";sequence <MLRecurrentNetworkActivation >activations ; };partial interface MLGraphBuilder {sequence <MLOperand >gru (MLOperand input ,MLOperand weight ,MLOperand recurrentWeight , [EnforceRange ]unsigned long steps , [EnforceRange ]unsigned long hiddenSize ,optional MLGruOptions options = {}); };dictionary {
MLGruSupportLimits MLSupportLimits input ;MLSupportLimits weight ;MLSupportLimits recurrentWeight ;MLSupportLimits bias ;MLSupportLimits recurrentBias ;MLSupportLimits initialHiddenState ;MLSupportLimits outputs ; };partial dictionary MLOpSupportLimits {MLGruSupportLimits gru ; };
MLGruOptions
has the following members:
bias
, of type MLOperand-
The 2-D input bias tensor of shape [numDirections, 3 * hiddenSize]. The ordering of the bias vectors in the second dimension of the tensor shape is specified according to the
layout
argument. recurrentBias
, of type MLOperand-
The 2-D recurrent bias tensor of shape [numDirections, 3 * hiddenSize]. The ordering of the bias vectors in the second dimension of the tensor shape is specified according to the
layout
argument. initialHiddenState
, of type MLOperand-
The 3-D initial hidden state tensor of shape [numDirections, batchSize, hiddenSize]. When not specified, implementations SHOULD use a tensor filled with zero.
resetAfter
, of type boolean, defaulting totrue
-
Indicates whether to apply the reset gate after or before matrix multiplication.
returnSequence
, of type boolean, defaulting tofalse
-
Indicates whether to also return the entire sequence with every output from each time step in it in addition to the output of the last time step.
direction
, of type MLRecurrentNetworkDirection, defaulting to"forward"
-
The processing direction of the input sequence. When set to
"both"
, the size of the first dimension of the weight and the bias tensor shapes must be 2, and the input is processed in both directions. layout
, of type MLGruWeightLayout, defaulting to"zrn"
-
The ordering of the weight and bias vectors for the internal gates of GRU, specifically the
update (z)
,reset (r)
, andnew (n)
gate, as indicated in the second dimension of the weight and bias tensor shape. activations
, of type sequence<MLRecurrentNetworkActivation>-
Specifies a pair of activation functions with the first function used for the update and reset gate, and the second used for the new gate. When not specified, defaults to the
"sigmoid"
and"tanh"
functions, respectively.
-
input
: anMLOperand
. The input 3-D tensor of shape [steps, batchSize, inputSize]. -
weight
: anMLOperand
. The 3-D input weight tensor of shape [numDirections, 3 * hiddenSize, inputSize]. The ordering of the weight vectors in the second dimension of the tensor shape is specified according to the options.layout
argument. -
recurrentWeight
: anMLOperand
. The 3-D recurrent weight tensor of shape [numDirections, 3 * hiddenSize, hiddenSize]. The ordering of the weight vectors in the second dimension of the tensor shape is specified according to the options.layout
argument. -
steps
: anunsigned long
scalar. The number of time steps in the recurrent network. The value must be greater than 0. -
hiddenSize
: anunsigned long
scalar. The value of the third dimension of the cell output tensor shape. It indicates the number of features in the hidden state. -
options
: an optionalMLGruOptions
. The optional parameters of the operation.
Returns: sequence<MLOperand
>. The first element is a 3-D tensor of shape [numDirections, batchSize, hiddenSize], the cell output from the last time step of the network. Additionally, if options.returnSequence
is set to true, the second element is the 4-D output tensor of shape [steps, numDirections, batchSize, hiddenSize] containing every cell outputs from each time step in the temporal sequence.
MLGruSupportLimits
has following members:
input
, of type MLSupportLimits-
MLSupportLimits
for input operand. weight
, of type MLSupportLimits-
MLSupportLimits
for weight operand. recurrentWeight
, of type MLSupportLimits-
MLSupportLimits
for recurrentWeight operand. bias
, of type MLSupportLimits-
MLSupportLimits
for bias operand. recurrentBias
, of type MLSupportLimits-
MLSupportLimits
for recurrentBias operand. initialHiddenState
, of type MLSupportLimits-
MLSupportLimits
for initialHiddenState operand. outputs
, of type MLSupportLimits-
MLSupportLimits
for all the output operands.
MLOpSupportLimits
has following member for gru()
:
gru
, of type MLGruSupportLimits-
Support limits for operator
gru()
.
The gru(input, weight, recurrentWeight, steps, hiddenSize, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of input, weight, recurrentWeight, options.
bias
(if it exists), options.recurrentBias
(if it exists), and options.(if it exists) returns false, then throw a
TypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
If the dataType of either weight or recurrentWeight is not equal to input’s dataType, then throw a
TypeError
. -
If input’s shape[0] is not equal to steps, then throw a
TypeError
. -
Let batchSize be input’s shape[1].
-
Let inputSize be input’s shape[2].
-
Let numDirections be 2 if options.
direction
is"both"
, or 1 otherwise. -
If weight’s shape is not equal to « numDirections, 3 * hiddenSize, inputSize », then throw a
TypeError
. -
If recurrentWeight’s shape is not equal to « numDirections, 3 * hiddenSize, hiddenSize », then throw a
TypeError
. -
If hiddenSize * 6 is not a valid dimension, then throw a
TypeError
.Why hiddenSize * 6 ?
Some underlying platforms operate on a single bias tensor which is a concatenation ofbias
andrecurrentBias
. Therefore, 3 * hiddenSize + 3 * hiddenSize must also be a valid dimension. -
If options.
recurrentBias
exists: -
If options.
exists:
-
If options.
activations
exists:-
Let activations be a clone of options.
activations
.
-
Otherwise:
-
Calculate the output shape:
-
Let desc0 be the result of creating an MLOperandDescriptor given input’s dataType and « numDirections, batchSize, hiddenSize ».
-
If options.
returnSequence
is true:-
Let desc1 be the result of creating an MLOperandDescriptor given input’s dataType and « steps, numDirections, batchSize, hiddenSize ».
-
-
-
Make graph connections:
-
Let operator be an operator for the "gru" operation, given weight, recurrentWeight, steps, hiddenSize and options.
-
Let output0 be the result of creating an MLOperand given this and desc0.
-
If options.
returnSequence
is true:-
Let output1 be the result of creating an MLOperand given this and desc1.
-
Let output be the list « output0, output1 ».
-
Set output0.
[[operator]]
and output1.[[operator]]
to operator.
-
-
Otherwise:
-
Let output be the list « output0 ».
-
Set output0.
[[operator]]
to operator.
-
-
Set operator’s inputs to input, weight, and recurrentWeight.
-
If options.
recurrentBias
exists, then add it to operator’s inputs. -
Set operator’s activation functions to a clone of activations.
-
Set operator’s output to output.
-
-
Return output.
Using a squeeze() helper, the behavior of this operation can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function gru( builder, input, weight, recurrentWeight, steps, hiddenSize, options) { const batchSize= input. shape()[ 1 ]; const inputSize= input. shape()[ 2 ]; const numDirections= ( options. direction== 'both' ? 2 : 1 ); let hiddenState= options. initialHiddenState; if ( ! hiddenState) { const desc= { dataType: 'float32' , shape: [ numDirections, 1 , hiddenSize] }; const totalSize= numDirections* hiddenSize; hiddenState= builder. constant( desc, new Float32Array( totalSize). fill( 0 )); } let sequence= null ; let currentWeight= []; let currentRecurrentWeight= []; let currentBias= []; let currentRecurrentBias= []; for ( let dir= 0 ; dir< numDirections; ++ dir) { currentWeight. push( squeeze( builder, builder. slice( weight, [ dir, 0 , 0 ], [ 1 , 3 * hiddenSize, inputSize]))); currentRecurrentWeight. push( squeeze( builder, builder. slice( recurrentWeight, [ dir, 0 , 0 ], [ 1 , 3 * hiddenSize, hiddenSize]))); currentBias. push( options. bias? ( squeeze( builder, builder. slice( options. bias, [ dir, 0 ], [ 1 , 3 * hiddenSize]))) : null ); currentRecurrentBias. push( options. recurrentBias? ( squeeze( builder, builder. slice( options. recurrentBias, [ dir, 0 ], [ 1 , 3 * hiddenSize]))) : null ); } for ( let step= 0 ; step< steps; ++ step) { let currentHidden= []; let currentOutput= null ; for ( let dir= 0 ; dir< numDirections; ++ dir) { currentHidden. push( squeeze( builder, builder. slice( hiddenState, [ dir, 0 , 0 ], [ 1 , batchSize, hiddenSize]))); } for ( let dir= 0 ; dir< numDirections; ++ dir) { let slice= ( dir== 1 || options. direction== 'backward' ? steps- step- 1 : step); let currentInput= squeeze( builder, builder. slice( input, [ slice, 0 , 0 ], [ 1 , batchSize, inputSize])); let result= builder. reshape( builder. gruCell( currentInput, currentWeight[ dir], currentRecurrentWeight[ dir], currentHidden[ dir], hiddenSize, { bias: currentBias[ dir], recurrentBias: currentRecurrentBias[ dir], resetAfter: options. resetAfter, layout: options. layout, activations: options. activations}), [ 1 , batchSize, hiddenSize]); currentOutput= ( currentOutput? builder. concat([ currentOutput, result], 0 ) : result); } hiddenState= currentOutput; if ( options. returnSequence) { currentOutput= builder. reshape( currentOutput, [ 1 , numDirections, batchSize, hiddenSize]); sequence= ( sequence? builder. concat([ sequence, currentOutput], 0 ) : currentOutput); } } return ( sequence? [ hiddenState, sequence] : [ hiddenState]); }
7.7.21. gruCell
A single time step of the Gated Recurrent Unit [GRU] recurrent network using an update gate and a reset gate to compute the hidden state that rolls into the output across the temporal sequence of a recurrent network.dictionary :
MLGruCellOptions MLOperatorOptions {MLOperand bias ;MLOperand recurrentBias ;boolean resetAfter =true ;MLGruWeightLayout layout = "zrn";sequence <MLRecurrentNetworkActivation >activations ; };partial interface MLGraphBuilder {MLOperand gruCell (MLOperand input ,MLOperand weight ,MLOperand recurrentWeight ,MLOperand hiddenState , [EnforceRange ]unsigned long hiddenSize ,optional MLGruCellOptions options = {}); };dictionary {
MLGruCellSupportLimits MLSupportLimits input ;MLSupportLimits weight ;MLSupportLimits recurrentWeight ;MLSupportLimits hiddenState ;MLSupportLimits bias ;MLSupportLimits recurrentBias ;MLSupportLimits output ; };partial dictionary MLOpSupportLimits {MLGruCellSupportLimits gruCell ; };
MLGruCellOptions
has the following members:
bias
, of type MLOperand-
The 1-D input bias tensor of shape [3 * hiddenSize]. The ordering of the bias vectors in the second dimension of the tensor shape is specified according to the
layout
argument. recurrentBias
, of type MLOperand-
The 1-D recurrent bias tensor of shape [3 * hiddenSize]. The ordering of the bias vectors in the second dimension of the tensor shape is specified according to the
layout
argument. resetAfter
, of type boolean, defaulting totrue
-
Indicates whether to apply the reset gate after or before matrix multiplication.
layout
, of type MLGruWeightLayout, defaulting to"zrn"
-
The ordering of the weight and bias vectors for the internal gates of GRU, specifically the
update (z)
,reset (r)
, andnew (n)
gate, as indicated in the second dimension of the weight and bias tensor shape. activations
, of type sequence<MLRecurrentNetworkActivation>-
Specifies a pair of activation functions with the first function used for the update and reset gate, and the second used for the new gate. When not specified, defaults to the
"sigmoid"
and"tanh"
functions, respectively.
-
input
: anMLOperand
. The input 2-D tensor of shape [batchSize, inputSize]. -
weight
: anMLOperand
. The 2-D input weight tensor of shape [3 * hiddenSize, inputSize]. The ordering of the weight vectors in the first dimension of the tensor shape is specified according to the options.layout argument. -
recurrentWeight
: anMLOperand
. The 2-D recurrent weight tensor of shape [3 * hiddenSize, hiddenSize]. The ordering of the weight vectors in the first dimension of the tensor shape is specified according to the options.layout argument. -
hiddenState
: anMLOperand
. The 2-D input hidden state tensor of shape [batchSize, hiddenSize]. -
hiddenSize
: anunsigned long
scalar. The value of the second dimension of the output tensor shape. It indicates the number of features in the hidden state. -
options
: an optionalMLGruCellOptions
. The optional parameters of the operation.
Returns: an MLOperand
. The 2-D tensor of shape [batchSize, hiddenSize], the cell output hidden state of a single time step of the recurrent network.
MLGruCellSupportLimits
has following members;
input
, of type MLSupportLimits-
MLSupportLimits
for input operand. weight
, of type MLSupportLimits-
MLSupportLimits
for weight operand. recurrentWeight
, of type MLSupportLimits-
MLSupportLimits
for recurrentWeight operand. hiddenState
, of type MLSupportLimits-
MLSupportLimits
for hiddenState operand. bias
, of type MLSupportLimits-
MLSupportLimits
for bias operand. recurrentBias
, of type MLSupportLimits-
MLSupportLimits
for recurrentBias operand. output
, of type MLSupportLimits-
MLSupportLimits
for output operand.
MLOpSupportLimits
has following member for gruCell()
:
gruCell
, of type MLGruCellSupportLimits-
Support limits for operator
gruCell()
.
The gruCell(input, weight, recurrentWeight, hiddenState, hiddenSize, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of input, weight, recurrentWeight, hiddenState, options.
bias
(if it exists), and options.recurrentBias
(if it exists) returns false, then throw aTypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
Let batchSize be input’s shape[0].
-
Let inputSize be input’s shape[1].
-
If the dataType of any of weight, recurrentWeight, or hiddenState is not equal to input’s dataType, then throw a
TypeError
. -
If weight’s shape is not equal to « 3 * hiddenSize, inputSize », then throw a
TypeError
. -
If recurrentWeight’s shape is not equal to « 3 * hiddenSize, hiddenSize », then throw a
TypeError
. -
If hiddenState’s shape is not equal to « batchSize, hiddenSize », then throw a
TypeError
. -
If hiddenSize * 6 is not a valid dimension, then throw a
TypeError
.Why hiddenSize * 6 ?
Some underlying platforms operate on a single bias tensor which is a concatenation ofbias
andrecurrentBias
. Therefore, 3 * hiddenSize + 3 * hiddenSize must also be a valid dimension. -
If options.
recurrentBias
exists: -
If options.
activations
exists:-
Let activations be a clone of options.
activations
.
-
Otherwise:
-
Let desc be the result of creating an MLOperandDescriptor given input’s dataType and « batchSize, hiddenSize ».
-
Make graph connections:
-
Let output be the result of creating an MLOperand given this and desc.
-
Let operator be an operator for the "gruCell" operation, given weight, recurrentWeight, hiddenState, hiddenSize and options.
-
Set output.
[[operator]]
to operator. -
Set operator’s inputs to input, weight, recurrentWeight, and hiddenState.
-
If options.
recurrentBias
exists, then add it to operator’s inputs. -
Set operator’s activation functions to a clone of activations.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation when the weight layout is the default "zrn"
layout, and the activation functions of the update/reset gate and new gate are sigmoid()
and tanh()
respectively can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function gruCell( builder, input, weight, recurrentWeight, hiddenState, hiddenSize, options) { const one= builder. constant( input. dataType(), 1 ); const zero= builder. constant( input. dataType(), 0 ); const inputSize= input. shape()[ 1 ]; // update gate (z) let z= builder. sigmoid( builder. add( builder. add( ( options. bias? builder. slice( options. bias, [ 0 ], [ hiddenSize]) : zero), ( options. recurrentBias? builder. slice( options. recurrentBias, [ 0 ], [ hiddenSize]) : zero)), builder. add( builder. matmul( input, builder. transpose( builder. slice( weight, [ 0 , 0 ], [ hiddenSize, inputSize]))), builder. matmul( hiddenState, builder. transpose( builder. slice( recurrentWeight, [ 0 , 0 ], [ hiddenSize, hiddenSize])))))); // reset gate (r) let r= builder. sigmoid( builder. add( builder. add( ( options. bias? builder. slice( options. bias, [ hiddenSize], [ hiddenSize]) : zero), ( options. recurrentBias? builder. slice( options. recurrentBias, [ hiddenSize], [ hiddenSize]) : zero)), builder. add( builder. matmul( input, builder. transpose( builder. slice( weight, [ hiddenSize, 0 ], [ hiddenSize, inputSize]))), builder. matmul( hiddenState, builder. transpose( builder. slice( recurrentWeight, [ hiddenSize, 0 ], [ hiddenSize, hiddenSize])))))); // new gate (n) let n; if ( options. resetAfter) { n= builder. tanh( builder. add( ( options. bias? builder. slice( options. bias, [ 2 * hiddenSize], [ hiddenSize]) : zero), builder. add( builder. matmul( input, builder. transpose( builder. slice( weight, [ 2 * hiddenSize, 0 ], [ hiddenSize, inputSize]))), builder. mul( r, builder. add( ( options. recurrentBias? builder. slice( options. recurrentBias, [ 2 * hiddenSize], [ hiddenSize]) : zero), builder. matmul( hiddenState, builder. transpose( builder. slice( recurrentWeight, [ 2 * hiddenSize, 0 ], [ hiddenSize, hiddenSize])))))))); } else { n= builder. tanh( builder. add( builder. add( ( options. bias? builder. slice( options. bias, [ 2 * hiddenSize], [ hiddenSize]) : zero), ( options. recurrentBias? builder. slice( options. recurrentBias, [ 2 * hiddenSize], [ hiddenSize]) : zero)), builder. add( builder. matmul( input, builder. transpose( builder. slice( weight, [ 2 * hiddenSize, 0 ], [ hiddenSize, inputSize]))), builder. matmul( builder. mul( r, hiddenState), builder. transpose( builder. slice( recurrentWeight, [ 2 * hiddenSize, 0 ], [ hiddenSize, hiddenSize])))))); } // compute the new hidden state return builder. add( builder. mul( z, hiddenState), builder. mul( n, builder. sub( one, z))); }
7.7.22. hardSigmoid
Calculate the non-smooth hard sigmoid function on the input tensor, used instead of the sigmoid function for faster computation.dictionary :
MLHardSigmoidOptions MLOperatorOptions {double alpha = 0.2;double beta = 0.5; };partial interface MLGraphBuilder {MLOperand hardSigmoid (MLOperand input ,optional MLHardSigmoidOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits hardSigmoid ; };
MLHardSigmoidOptions
has the following members:
alpha
, of type double, defaulting to0.2
-
A scalar multiplier.
beta
, of type double, defaulting to0.5
-
A scalar addition.
-
input
: anMLOperand
. The input tensor. -
options
: an optionalMLHardSigmoidOptions
. The optional parameters of the operation.
Returns:
-
an
MLOperand
. The output tensor of the same shape as input.
MLOpSupportLimits
has following member for hardSigmoid()
:
hardSigmoid
, of type MLSingleInputSupportLimits-
Support limits for operator
hardSigmoid()
.
The hardSigmoid(input, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
Set options.
alpha
to the result of casting options.alpha
to input’s dataType. -
Set options.
beta
to the result of casting options.beta
to input’s dataType. -
Make graph connections:
-
Let output be the result of copying an MLOperand given input.
-
Let operator be an operator for the "hardSigmoid" operation, given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function hardSigmoid( builder, input, options) { return builder. max( builder. min( builder. add( builder. mul( builder. constant( input. dataType(), options. alpha), input), builder. constant( input. dataType(), options. beta)), builder. constant( input. dataType(), 1 )), builder. constant( input. dataType(), 0 )); }
7.7.23. hardSwish
Computes the nonlinear functiony = x * max(0, min(6, (x + 3))) / 6
that is introduced by [MobileNetV3] on the input tensor element-wise.
partial interface MLGraphBuilder {MLOperand hardSwish (MLOperand input ,optional MLOperatorOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits hardSwish ; };
-
input
: anMLOperand
. The input tensor. -
options
: anMLOperatorOptions
. Specifies the optional parameters of the operation.
Returns:
-
an
MLOperand
. The output tensor of the same shape as input.
MLOpSupportLimits
has following member for hardSwish()
:
hardSwish
, of type MLSingleInputSupportLimits-
Support limits for operator
hardSwish()
.
The hardSwish(input, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
Make graph connections:
-
Let output be the result of copying an MLOperand given input.
-
Let operator be an operator for the "hardSwish" operation, given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function hardSwish( builder, input, options) { return builder. div( builder. mul( input, builder. max( builder. constant( input. dataType(), 0 ), builder. min( builder. constant( input. dataType(), 6 ), builder. add( input, builder. constant( input. dataType(), 3 ))))), builder. constant( input. dataType(), 6 )); }
7.7.24. instanceNormalization
Normalize the input using [Instance-Normalization]. UnlikebatchNormalization()
where the mean and variance values used in the normalization are computed across all the samples in the batch dimension while the model is trained, the mean and variance values used in the instance normalization are computed on the fly for each input feature of each individual sample in the batch.
dictionary :
MLInstanceNormalizationOptions MLOperatorOptions {MLOperand scale ;MLOperand bias ;double epsilon = 1e-5;MLInputOperandLayout layout = "nchw"; };partial interface MLGraphBuilder {MLOperand instanceNormalization (MLOperand input ,optional MLInstanceNormalizationOptions options = {}); };dictionary {
MLNormalizationSupportLimits MLSupportLimits input ;MLSupportLimits scale ;MLSupportLimits bias ;MLSupportLimits output ; };partial dictionary MLOpSupportLimits {MLNormalizationSupportLimits instanceNormalization ; };
MLInstanceNormalizationOptions
has the following members:
scale
, of type MLOperand-
The 1-D tensor of the scaling values whose size is equal to the number of channels, i.e. the size of the feature dimension of the input. For example, for an input tensor with
"nchw"
layout, the size is equal to input’s shape[1]. bias
, of type MLOperand-
The 1-D tensor of the bias values whose size is equal to the size of the feature dimension of the input. For example, for an input tensor with
"nchw"
layout, the size is equal to input’s shape[1]. epsilon
, of type double, defaulting to1e-5
-
A small value to prevent computational error due to divide-by-zero.
layout
, of type MLInputOperandLayout, defaulting to"nchw"
-
The layout format of the input.
-
input
: anMLOperand
. The input 4-D tensor. -
options
: an optionalMLInstanceNormalizationOptions
. The optional parameters of the operation.
Returns: an MLOperand
. The instance-normalized 4-D tensor of the same shape as input.
MLNormalizationSupportLimits
has following members:
input
, of type MLSupportLimits-
MLSupportLimits
for input operand. scale
, of type MLSupportLimits-
MLSupportLimits
for scale operand. bias
, of type MLSupportLimits-
MLSupportLimits
for bias operand. output
, of type MLSupportLimits-
MLSupportLimits
for output operand.
MLOpSupportLimits
has following member for instanceNormalization()
:
instanceNormalization
, of type MLNormalizationSupportLimits-
Support limits for operator
instanceNormalization()
.
The instanceNormalization(input, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of input, options.
scale
(if it exists), and options.bias
(if it exists) returns false, then throw aTypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
Set options.
epsilon
to the result of casting options.epsilon
to input’s dataType. -
Make graph connections:
-
Let output be the result of copying an MLOperand given input.
-
Let operator be an operator for the "instanceNormalization" operation, given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation when the input tensor is 4-D of the "nchw"
layout can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function instanceNormalization( builder, input, options) { // The reduction of the mean and variance values happens over the spatial // dimensions of the input e.g. axis 2 and 3 of the input tensor. const reduceOptions= { axes: [ 2 , 3 ], keepDimensions: true }; const mean= builder. reduceMean( input, reduceOptions); const variance= builder. reduceMean( builder. pow( builder. sub( input, mean), builder. constant( input. dataType(), 2 )), reduceOptions); // The scale and bias values are applied per input feature // e.g. axis 1 of the input tensor. const shape= [ 1 , input. shape()[ 1 ], 1 , 1 ]; return builder. add( builder. mul( builder. reshape( options. scale, shape), builder. div( builder. sub( input, mean), builder. sqrt( builder. add( variance, options. epsilon)))), builder. reshape( options. bias, shape)); }
7.7.25. layerNormalization
Normalize the input using [Layer-Normalization]. UnlikebatchNormalization()
where the mean and variance values are computed across all the samples in the batch dimension while the model is trained, and in instanceNormalization()
where the mean and variance values are computed on the fly for each input feature of each individual sample in the batch, the means and variance values of the layer normalization are computed on the fly across all the input features of each individual sample in the batch.
dictionary :
MLLayerNormalizationOptions MLOperatorOptions {MLOperand scale ;MLOperand bias ;sequence <[EnforceRange ]unsigned long >axes ;double epsilon = 1e-5; };partial interface MLGraphBuilder {MLOperand layerNormalization (MLOperand input ,optional MLLayerNormalizationOptions options = {}); };partial dictionary MLOpSupportLimits {MLNormalizationSupportLimits layerNormalization ; };
MLLayerNormalizationOptions
has the following members:
scale
, of type MLOperand-
The N-D tensor of the scaling values whose shape is determined by the axes member in that each value in axes indicates the dimension of the input tensor with scaling values. For example, for an axes values of [1,2,3], the shape of this tensor is the list of the corresponding sizes of the input dimension 1, 2 and 3. When this member is not present, the scaling value is assumed to be 1.
bias
, of type MLOperand-
The N-D tensor of the bias values whose shape is determined by the axes member in that each value in axes indicates the dimension of the input tensor with bias values. For example, for an axes values of [1,2,3], the shape of this tensor is the list of the corresponding sizes of the input dimension 1, 2 and 3. When this member is not present, the bias value is assumed to be 0.
axes
, of typesequence<[EnforceRange] unsigned long>
-
The indices to the input dimensions to reduce. When this member is not present, it is treated as if all dimensions except the first were given (e.g. for a 4-D input tensor, axes = [1,2,3]). That is, the reduction for the mean and variance values are calculated across all the input features for each independent batch. If empty, no dimensions are reduced.
epsilon
, of type double, defaulting to1e-5
-
A small value to prevent computational error due to divide-by-zero.
-
input
: anMLOperand
. The input N-D tensor. -
options
: an optionalMLLayerNormalizationOptions
. The optional parameters of the operation.
Returns: an MLOperand
. The layer-normalized N-D tensor of the same shape as input.
MLOpSupportLimits
has following member for layerNormalization()
:
layerNormalization
, of type MLNormalizationSupportLimits-
Support limits for operator
layerNormalization()
.
The layerNormalization(input, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of input, options.
scale
(if it exists), and options.bias
(if it exists) returns false, then throw aTypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
If options.
axes
does not exist, then set options.axes
to a new list, either equal to the range from 1 to input’s rank, exclusive, if input’s rank is greater than 1, or an empty list otherwise. -
Otherwise, if options.
axes
contains duplicate values, or if any of its elements is not in the range 0 to input’s rank, exclusive, then return failure. -
Set options.
epsilon
to the result of casting options.epsilon
to input’s dataType. -
For each index in the range 0 to options.
axes
's size, exclusive: -
Make graph connections:
-
Let output be the result of copying an MLOperand given input.
-
Let operator be an operator for the "layerNormalization" operation, given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation when the axes parameter is set to [1,2,3] can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function layerNormalization( builder, input, options) { // The reduction of the mean and variance values happens over the spatial // dimensions across all the input features (i.e. all channels) of the input // tensor. const reduceOptions= { axes: [ 1 , 2 , 3 ], keepDimensions: true }; const mean= builder. reduceMean( input, reduceOptions); const variance= builder. reduceMean( builder. pow( builder. sub( input, mean), builder. constant( input. dataType(), 2 )), reduceOptions); // The scale and bias tensors are of the shape of the input // specified by the values in the axes parameter (i.e. [1,2,3]). return builder. add( builder. mul( options. scale, builder. div( builder. sub( input, mean), builder. sqrt( builder. add( variance, options. epsilon)))), options. bias); }
7.7.26. leakyRelu
Calculate the leaky version of rectified linear function on the input tensor element-wise. The calculation follows the expressionmax(0, x) + alpha * min(0, x)
.
dictionary :
MLLeakyReluOptions MLOperatorOptions {double alpha = 0.01; };partial interface MLGraphBuilder {MLOperand leakyRelu (MLOperand input ,optional MLLeakyReluOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits leakyRelu ; };
MLLeakyReluOptions
has the following members:
alpha
, of type double, defaulting to0.01
-
A scalar multiplier.
-
input
: anMLOperand
. The input tensor. -
options
: an optionalMLLeakyReluOptions
. The optional parameters of the operation.
Returns:
-
an
MLOperand
. The output tensor of the same shape as input.
MLOpSupportLimits
has following member for leakyRelu()
:
leakyRelu
, of type MLSingleInputSupportLimits-
Support limits for operator
leakyRelu()
.
The leakyRelu(input, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
Set options.
alpha
to the result of casting options.alpha
to input’s dataType. -
Make graph connections:
-
Let output be the result of copying an MLOperand given input.
-
Let operator be an operator for the "leakyRelu" operation, given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function leakyRelu( builder, input, options) { return builder. add( builder. max( builder. constant( input. dataType(), 0 ), input), builder. mul( builder. constant( input. dataType(), options. alpha), builder. min( builder. constant( input. dataType(), 0 ), input))); }
7.7.27. linear
Calculate a linear functiony = alpha * x + beta
on the input tensor.
dictionary :
MLLinearOptions MLOperatorOptions {double alpha = 1;double beta = 0; };partial interface MLGraphBuilder {MLOperand linear (MLOperand input ,optional MLLinearOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits linear ; };
MLLinearOptions
has the following members:
alpha
, of type double, defaulting to1
-
A scalar multiplier.
beta
, of type double, defaulting to0
-
A scalar addition.
-
input
: anMLOperand
. The input tensor. -
options
: an optionalMLLinearOptions
. The optional parameters of the operation.
Returns:
-
an
MLOperand
. The output tensor of the same shape as input.
MLOpSupportLimits
has following member for linear()
:
linear
, of type MLSingleInputSupportLimits-
Support limits for operator
linear()
.
The linear(input, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
Set options.
alpha
to the result of casting options.alpha
to input’s dataType. -
Set options.
beta
to the result of casting options.beta
to input’s dataType. -
Make graph connections:
-
Let output be the result of copying an MLOperand given input.
-
Let operator be an operator for the "linear" operation, given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function linear( builder, input, options) { return builder. add( builder. mul( input, builder. constant( input. dataType(), options. alpha)), builder. constant( input. dataType(), options. beta)); }
7.7.28. lstm
Long Short-Term Memory [LSTM] recurrent network uses an input, output, forget, and cell gate to compute the output state that rolls into the output across the temporal sequence of the network.enum {
MLLstmWeightLayout , // input-output-forget-cell gate ordering
"iofg" // input-forget-cell-output gate ordering };
"ifgo" dictionary :
MLLstmOptions MLOperatorOptions {MLOperand bias ;MLOperand recurrentBias ;MLOperand peepholeWeight ;MLOperand initialHiddenState ;MLOperand initialCellState ;boolean returnSequence =false ;MLRecurrentNetworkDirection direction = "forward";MLLstmWeightLayout layout = "iofg";sequence <MLRecurrentNetworkActivation >activations ; };partial interface MLGraphBuilder {sequence <MLOperand >lstm (MLOperand input ,MLOperand weight ,MLOperand recurrentWeight , [EnforceRange ]unsigned long steps , [EnforceRange ]unsigned long hiddenSize ,optional MLLstmOptions options = {}); };dictionary {
MLLstmSupportLimits MLSupportLimits input ;MLSupportLimits weight ;MLSupportLimits recurrentWeight ;MLSupportLimits bias ;MLSupportLimits recurrentBias ;MLSupportLimits peepholeWeight ;MLSupportLimits initialHiddenState ;MLSupportLimits initialCellState ;MLSupportLimits outputs ; };partial dictionary MLOpSupportLimits {MLLstmSupportLimits lstm ; };
MLLstmOptions
has the following members:
bias
, of type MLOperand-
The 2-D input bias tensor of shape [numDirections, 4 * hiddenSize]. The ordering of the bias vectors in the second dimension of the tensor shape is specified according to
layout
. recurrentBias
, of type MLOperand-
The 2-D recurrent bias tensor of shape [numDirections, 4 * hiddenSize]. The ordering of the bias vectors in the first dimension of the tensor shape is specified according to
layout
. peepholeWeight
, of type MLOperand-
The 2-D weight tensor for peepholes of shape [numDirections, 3 * hiddenSize]. The pack ordering of the weight vectors is for the
input (i)
,output (o)
, andforget (f)
gate, respectively. initialHiddenState
, of type MLOperand-
The 3-D initial hidden state tensor of shape [numDirections, batchSize, hiddenSize]. When not specified, implementations SHOULD use a tensor filled with zero.
initialCellState
, of type MLOperand-
The 3-D initial hidden state tensor of shape [numDirections, batchSize, hiddenSize]. When not specified, implementations SHOULD use a tensor filled with zero.
returnSequence
, of type boolean, defaulting tofalse
-
Indicates whether to also return the entire sequence with every output from each time step in it in addition to the output of the last time step.
direction
, of type MLRecurrentNetworkDirection, defaulting to"forward"
-
The processing direction of the input sequence. When set to
"both"
, the size of the first dimension of the weight and the bias tensor shapes must be 2, and the input is processed in both directions. layout
, of type MLLstmWeightLayout, defaulting to"iofg"
-
The ordering of the weight and bias vectors for the internal gates of LSTM, specifically the
input (i)
,output (o)
,forget (f)
, andcell (g)
gate, as indicated in the first dimension of the weight and bias tensor shapes. activations
, of type sequence<MLRecurrentNetworkActivation>-
A list of three activation functions, the first one is used for the
input (i)
,forget (f)
, andoutput (o)
gate, the second one is used for thecell (g)
gate, and the last used for filtering the output cell state before combining it with the result of the output gate to form the output hidden state. When not specified, defaults to a sequence of the"sigmoid"
,"tanh"
, and"tanh"
functions, respectively.
-
input
: anMLOperand
. The input 3-D tensor of shape [steps, batchSize, inputSize]. -
weight
: anMLOperand
. The 3-D input weight tensor of shape [numDirections, 4 * hiddenSize, inputSize]. The ordering of the weight vectors in the second dimension of the tensor shape is specified according to the options.layout
. -
recurrentWeight
: anMLOperand
. The 3-D recurrent weight tensor of shape [numDirections, 4 * hiddenSize, hiddenSize]. The ordering of the weight vectors in the second dimension of the tensor shape is specified according to the options.layout
argument. -
steps
: anunsigned long
scalar. The number of time steps in the recurrent network. The value must be greater than 0. -
hiddenSize
: anunsigned long
scalar. The value of the third dimension of the cell output tensor shape. It indicates the number of features in the hidden state. -
options
: an optionalMLLstmOptions
. The optional parameters of the operation.
Returns: sequence<MLOperand
>. The first element is a 3-D tensor of shape [numDirections, batchSize, hiddenSize], the output hidden state from the last time step of the network. The second element is a 3-D tensor of shape [numDirections, batchSize, hiddenSize], the output cell state from the last time step of the network. Additionally, if options.returnSequence
is set to true, the third element is the 4-D output tensor of shape [steps, numDirections, batchSize, hiddenSize] containing every output from each time step in the temporal sequence.
MLLstmSupportLimits
has following members:
input
, of type MLSupportLimits-
MLSupportLimits
for input operand. weight
, of type MLSupportLimits-
MLSupportLimits
for weight operand. recurrentWeight
, of type MLSupportLimits-
MLSupportLimits
for recurrentWeight operand. bias
, of type MLSupportLimits-
MLSupportLimits
for bias operand. recurrentBias
, of type MLSupportLimits-
MLSupportLimits
for recurrentBias operand. peepholeWeight
, of type MLSupportLimits-
MLSupportLimits
for peepholeWeight operand. initialHiddenState
, of type MLSupportLimits-
MLSupportLimits
for initialHiddenState operand. initialCellState
, of type MLSupportLimits-
MLSupportLimits
for initialCellState operand. outputs
, of type MLSupportLimits-
MLSupportLimits
for all the output operands.
MLOpSupportLimits
has following member for lstm()
:
lstm
, of type MLLstmSupportLimits-
Support limits for operator
lstm()
.
The lstm(input, weight, recurrentWeight, steps, hiddenSize, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of input, weight, recurrentWeight, options.
bias
(if it exists), options.recurrentBias
(if it exists), options.peepholeWeight
(if it exists), options.(if it exists), and options.
initialCellState
(if it exists) returns false, then throw aTypeError
. -
Let numDirections be 2 if options.
direction
is"both"
, or 1 otherwise. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
If input’s shape[0] is not equal to steps, then throw a
TypeError
. -
If the dataType of either weight or recurrentWeight is not equal to input’s dataType, then throw a
TypeError
. -
Let batchSize be input’s shape[1].
-
Let inputSize be input’s shape[2].
-
If the dataType of either weight or recurrentWeight is not equal to input’s dataType, then throw a
TypeError
. -
If weight’s shape is not equal to « numDirections, 4 * hiddenSize, inputSize », then throw a
TypeError
. -
If recurrentWeight’s shape is not equal to « numDirections, 4 * hiddenSize, hiddenSize », then throw a
TypeError
. -
If hiddenSize * 8 is not a valid dimension, then throw a
TypeError
.Why hiddenSize * 8 ?
Some underlying platforms operate on a single bias tensor which is a concatenation ofbias
andrecurrentBias
. Therefore, 4 * hiddenSize + 4 * hiddenSize must also be a valid dimension. -
If options.
recurrentBias
exists: -
If options.
peepholeWeight
exists: -
If options.
exists:
-
If options.
initialCellState
exists: -
If options.
activations
exists:-
Let activations be a clone of options.
activations
.
-
Otherwise:
-
Calculate the output shape:
-
Let desc be the result of creating an MLOperandDescriptor given input’s dataType and « numDirections, batchSize, hiddenSize ».
-
If options.
returnSequence
is true:-
Let desc2 be the result of creating an MLOperandDescriptor given input’s dataType and « steps, numDirections, batchSize, hiddenSize ».
-
-
-
Make graph connections:
-
Let operator be an operator for the "lstm" operation, given weight, recurrentWeight, steps, hiddenSize and options.
-
Let output0 be the result of creating an MLOperand given this and desc.
-
Let output1 be the result of creating an MLOperand given this and desc.
-
If options.
returnSequence
is true:-
Let output2 be the result of creating an MLOperand given this and desc2.
-
Let output be the list « output0, output1, output2 ».
-
Set output0.
[[operator]]
, output1.[[operator]]
and output2.[[operator]]
to operator.
-
-
Otherwise:
-
Let output be the list « output0, output1 ».
-
Set output0.
[[operator]]
and output1.[[operator]]
to operator.
-
-
Set operator’s inputs to input, weight, and recurrentWeight.
-
If options.
recurrentBias
exists, then add it to operator’s inputs. -
If options.
peepholeWeight
exists, then add it to operator’s inputs. -
If options.
initialCellState
exists, then add it to operator’s inputs. -
Set operator’s activation functions to a clone of activations.
-
Set operator’s output to output.
-
-
Return output.
Using a squeeze() helper, the behavior of this operation can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function lstm( builder, input, weight, recurrentWeight, steps, hiddenSize, options) { const batchSize= input. shape()[ 1 ]; const inputSize= input. shape()[ 2 ]; const numDirections= ( options. direction== 'both' ? 2 : 1 ); let hiddenState= options. initialHiddenState; let cellState= options. initialCellState; if ( ! hiddenState) { const desc= { dataType: 'float32' , shape: [ numDirections, 1 , hiddenSize] }; const totalSize= numDirections* hiddenSize; hiddenState= builder. constant( desc, new Float32Array( totalSize). fill( 0 )); } if ( ! cellState) { const desc= { dataType: 'float32' , shape: [ numDirections, 1 , hiddenSize] }; const totalSize= numDirections* hiddenSize; cellState= builder. constant( desc, new Float32Array( totalSize). fill( 0 )); } let sequence= null ; let currentWeight= []; let currentRecurrentWeight= []; let currentBias= []; let currentRecurrentBias= []; let currentPeepholeWeight= []; for ( let dir= 0 ; dir< numDirections; ++ dir) { currentWeight. push( squeeze( builder, builder. slice( weight, [ dir, 0 , 0 ], [ 1 , 4 * hiddenSize, inputSize]))); currentRecurrentWeight. push( squeeze( builder, builder. slice( recurrentWeight, [ dir, 0 , 0 ], [ 1 , 4 * hiddenSize, hiddenSize]))); currentBias. push( options. bias? ( squeeze( builder, builder. slice( options. bias, [ dir, 0 ], [ 1 , 4 * hiddenSize]))) : null ); currentRecurrentBias. push( options. recurrentBias? ( squeeze( builder, builder. slice( options. recurrentBias, [ dir, 0 ], [ 1 , 4 * hiddenSize]))) : null ); currentPeepholeWeight. push( options. peepholeWeight? ( squeeze( builder, builder. slice( options. peepholeWeight, [ dir, 0 ], [ 1 , 3 * hiddenSize]))) : null ); } for ( let step= 0 ; step< steps; ++ step) { let currentHidden= []; let currentCell= []; let nextHidden= null ; let nextCell= null ; for ( let dir= 0 ; dir< numDirections; ++ dir) { currentHidden. push( squeeze( builder, builder. slice( hiddenState, [ dir, 0 , 0 ], [ 1 , batchSize, hiddenSize]))); currentCell. push( squeeze( builder, builder. slice( cellState, [ dir, 0 , 0 ], [ 1 , batchSize, hiddenSize]))); } for ( let dir= 0 ; dir< numDirections; ++ dir) { let slice= ( dir== 1 || options. direction== 'backward' ? steps- step- 1 : step); let currentInput= squeeze( builder, builder. slice( input, [ slice, 0 , 0 ], [ 1 , batchSize, inputSize])); let results= builder. lstmCell( currentInput, currentWeight[ dir], currentRecurrentWeight[ dir], currentHidden[ dir], currentCell[ dir], hiddenSize, { bias: currentBias[ dir], recurrentBias: currentRecurrentBias[ dir], peepholeWeight: currentPeepholeWeight[ dir], layout: options. layout, activations: options. activations}); let output= builder. reshape( results[ 0 ], [ 1 , batchSize, hiddenSize]); let cell= builder. reshape( results[ 1 ], [ 1 , batchSize, hiddenSize]); nextHidden= ( nextHidden? builder. concat([ nextHidden, output], 0 ) : output); nextCell= ( nextCell? builder. concat([ nextCell, cell], 0 ) : cell); } hiddenState= nextHidden; cellState= nextCell; if ( options. returnSequence) { nextHidden= builder. reshape( nextHidden, [ 1 , numDirections, batchSize, hiddenSize]); sequence= ( sequence? builder. concat([ sequence, nextHidden], 0 ) : nextHidden); } } return ( sequence? [ hiddenState, cellState, sequence] : [ hiddenState, cellState]); }
7.7.29. lstmCell
A single time step of the Long Short-Term Memory [LSTM] recurrent network using a cell state, an input, output, and forget gate to compute the cell state and the hidden state of the next time step that rolls into the output across the temporal sequence of the network.dictionary :
MLLstmCellOptions MLOperatorOptions {MLOperand bias ;MLOperand recurrentBias ;MLOperand peepholeWeight ;MLLstmWeightLayout layout = "iofg";sequence <MLRecurrentNetworkActivation >activations ; };partial interface MLGraphBuilder {sequence <MLOperand >lstmCell (MLOperand input ,MLOperand weight ,MLOperand recurrentWeight ,MLOperand hiddenState ,MLOperand cellState , [EnforceRange ]unsigned long hiddenSize ,optional MLLstmCellOptions options = {}); };dictionary {
MLLstmCellSupportLimits MLSupportLimits input ;MLSupportLimits weight ;MLSupportLimits recurrentWeight ;MLSupportLimits hiddenState ;MLSupportLimits cellState ;MLSupportLimits bias ;MLSupportLimits recurrentBias ;MLSupportLimits peepholeWeight ;MLSupportLimits outputs ; };partial dictionary MLOpSupportLimits {MLLstmCellSupportLimits lstmCell ; };
MLLstmCellOptions
has the following members:
bias
, of type MLOperand-
The 1-D input bias tensor of shape [4 * hiddenSize]. The ordering of the bias vectors in the first dimension of the tensor shape is specified according to the
layout
argument. recurrentBias
, of type MLOperand-
The 1-D recurrent bias tensor of shape [4 * hiddenSize]. The ordering of the bias vectors in the first dimension of the tensor shape is specified according to the
layout
argument. peepholeWeight
, of type MLOperand-
The 1-D weight tensor for peepholes of shape [3 * hiddenSize]. The pack ordering of the weight vectors is for the
input (i)
,output (o)
, andforget (f)
gate, respectively. layout
, of type MLLstmWeightLayout, defaulting to"iofg"
-
The ordering of the weight and bias vectors for the internal gates of LSTM, specifically the
input (i)
,output (o)
,forget (f)
, andcell (g)
gate, as indicated in the first dimension of the weight and bias tensor shapes. activations
, of type sequence<MLRecurrentNetworkActivation>-
A list of three activation functions, the first one is used for the
input (i)
,forget (f)
, andoutput (o)
gate, the second one is used for thecell (g)
gate, and the last used for filtering the output cell state before combining it with the result of the output gate to form the output hidden state. When not specified, defaults to a sequence of the"sigmoid"
,"tanh"
, and"tanh"
functions, respectively.
-
input
: anMLOperand
. The input 2-D tensor of shape [batchSize, inputSize]. -
weight
: anMLOperand
. The 2-D input weight tensor of shape [4 * hiddenSize, inputSize]. The ordering of the weight vectors in the first dimension of the tensor shape is specified according to the options.layout argument. -
recurrentWeight
: anMLOperand
. The 2-D recurrent weight tensor of shape [4 * hiddenSize, hiddenSize]. The ordering of the weight vectors in the first dimension of the tensor shape is specified according to the options.layout argument. -
hiddenState
: anMLOperand
. The 2-D input hidden state tensor of shape [batchSize, hiddenSize]. -
cellState
: anMLOperand
. The 2-D input cell state tensor of shape [batchSize, hiddenSize]. -
hiddenSize
: anunsigned long
scalar. The value of the second dimension of the output tensor shape. It indicates the number of features in the hidden state. -
options
: an optionalMLLstmCellOptions
. The optional parameters of the operation.
Returns: sequence<MLOperand
>. The first element is the output hidden state of the current time step of the recurrent network. The following element is the output cell state. Both elements are 2-D tensors of shape [batchSize, hiddenSize].
MLLstmCellSupportLimits
has following members:
input
, of type MLSupportLimits-
MLSupportLimits
for input operand. weight
, of type MLSupportLimits-
MLSupportLimits
for weight operand. recurrentWeight
, of type MLSupportLimits-
MLSupportLimits
for recurrentWeight operand. hiddenState
, of type MLSupportLimits-
MLSupportLimits
for hiddenState operand. cellState
, of type MLSupportLimits-
MLSupportLimits
for cellState operand. bias
, of type MLSupportLimits-
MLSupportLimits
for bias operand. recurrentBias
, of type MLSupportLimits-
MLSupportLimits
for recurrentBias operand. peepholeWeight
, of type MLSupportLimits-
MLSupportLimits
for peepholeWeight operand. outputs
, of type MLSupportLimits-
MLSupportLimits
for all the output operands.
MLOpSupportLimits
has following member for lstmCell()
:
lstmCell
, of type MLLstmCellSupportLimits-
Support limits for operator
lstmCell()
.
The lstmCell(input, weight, recurrentWeight, hiddenState, cellState, hiddenSize, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of input, weight, recurrentWeight, hiddenState, cellState, options.
bias
(if it exists), options.recurrentBias
(if it exists), and options.peepholeWeight
(if it exists) returns false, then throw aTypeError
. -
If input’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
If the dataType of any of weight, recurrentWeight, hiddenState or cellState is not equal to input’s dataType, then throw a
TypeError
. -
Let batchSize be input’s shape[0].
-
Let inputSize be input’s shape[1].
-
If weight’s shape is not equal to « 4 * hiddenSize, inputSize », then throw a
TypeError
. -
If recurrentWeight’s shape is not equal to « 4 * hiddenSize, hiddenSize », then throw a
TypeError
. -
If hiddenState’s shape is not equal to « batchSize, hiddenSize », then throw a
TypeError
. -
If cellState’s shape is not equal to « batchSize, hiddenSize », then throw a
TypeError
. -
If hiddenSize * 8 is not a valid dimension, then throw a
TypeError
.Why hiddenSize * 8 ?
Some underlying platforms operate on a single bias tensor which is a concatenation ofbias
andrecurrentBias
. Therefore, 4 * hiddenSize + 4 * hiddenSize must also be a valid dimension. -
If options.
recurrentBias
exists: -
If options.
peepholeWeight
exists: -
If options.
activations
exists:-
Let activations be a clone of options.
activations
.
-
Otherwise:
-
Let desc be a new
MLOperandDescriptor
. -
Make graph connections:
-
Let output0 be the result of creating an MLOperand given this and desc.
-
Let output1 be the result of creating an MLOperand given this and desc.
-
Let output be the list « output0, output1 ».
-
Let operator be an operator for the "lstmCell" operation, given weight, recurrentWeight, hiddenState, cellState, hiddenSize and options.
-
Set output0.
[[operator]]
and output1.[[operator]]
to operator. -
Set operator’s inputs to input, weight, recurrentWeight, hiddenState, and cellState.
-
If options.
recurrentBias
exists, then add it to operator’s inputs. -
If options.
peepholeWeight
exists, then add it to operator’s inputs. -
Set operator’s activation functions to a clone of activations.
-
Set operator’s output to output.
-
-
Return output.
The behavior of this operation when the weight layout is the default "iofg"
layout, and the activation functions of the input/forget/output gate and the cell gate/the cell state’s filter for the output hidden state are sigmoid()
and tanh()
respectively can be generically emulated from the usage of other operations as follows, although user agents typically have a more efficient implementation. In cases where the underlying platform does not directly support an operation, this decomposition can be used as a template to guide the implementation.
function lstmCell( builder, input, weight, recurrentWeight, hiddenState, cellState, hiddenSize, options) { const zero= builder. constant( input. dataType(), 0 ); const inputSize= input. shape()[ 1 ]; // input gate (i) let i= builder. sigmoid( builder. add( builder. mul( cellState, ( options. peepholeWeight? builder. slice( options. peepholeWeight, [ 0 ], [ hiddenSize]) : zero)), builder. add( builder. add( ( options. bias? builder. slice( options. bias, [ 0 ], [ hiddenSize]) : zero), ( options. recurrentBias? builder. slice( options. recurrentBias, [ 0 ], [ hiddenSize]) : zero)), builder. add( builder. matmul( input, builder. transpose( builder. slice( weight, [ 0 , 0 ], [ hiddenSize, inputSize]))), builder. matmul( hiddenState, builder. transpose( builder. slice( recurrentWeight, [ 0 , 0 ], [ hiddenSize, hiddenSize]))))))); // forget gate (f) let f= builder. sigmoid( builder. add( builder. mul( cellState, ( options. peepholeWeight? builder. slice( options. peepholeWeight, [ 2 * hiddenSize], [ hiddenSize]) : zero)), builder. add( builder. add( ( options. bias? builder. slice( options. bias, [ 2 * hiddenSize], [ hiddenSize]) : zero), ( options. recurrentBias? builder. slice( options. recurrentBias, [ 2 * hiddenSize], [ hiddenSize]) : zero)), builder. add( builder. matmul( input, builder. transpose( builder. slice( weight, [ 2 * hiddenSize, 0 ], [ hiddenSize, inputSize]))), builder. matmul( hiddenState, builder. transpose( builder. slice( recurrentWeight, [ 2 * hiddenSize, 0 ], [ hiddenSize, hiddenSize]))))))); // cell gate (g) let g= builder. tanh( builder. add( builder. add( ( options. bias? builder. slice( options. bias, [ 3 * hiddenSize], [ hiddenSize]) : zero), ( options. recurrentBias? builder. slice( options. recurrentBias, [ 3 * hiddenSize], [ hiddenSize]) : zero)), builder. add( builder. matmul( input, builder. transpose( builder. slice( weight, [ 3 * hiddenSize, 0 ], [ hiddenSize, inputSize]))), builder. matmul( hiddenState, builder. transpose( builder. slice( recurrentWeight, [ 3 * hiddenSize, 0 ], [ hiddenSize, hiddenSize])))))); // output gate (o) let o= builder. sigmoid( builder. add( builder. mul( cellState, ( options. peepholeWeight? builder. slice( options. peepholeWeight, [ hiddenSize], [ hiddenSize]) : zero)), builder. add( builder. add( ( options. bias? builder. slice( options. bias, [ hiddenSize], [ hiddenSize]) : zero), ( options. recurrentBias? builder. slice( options. recurrentBias, [ hiddenSize], [ hiddenSize]) : zero)), builder. add( builder. matmul( input, builder. transpose( builder. slice( weight, [ hiddenSize, 0 ], [ hiddenSize, inputSize]))), builder. matmul( hiddenState, builder. transpose( builder. slice( recurrentWeight, [ hiddenSize, 0 ], [ hiddenSize, hiddenSize]))))))); // output cell state (ct) let ct= builder. add( builder. mul( f, cellState), builder. mul( i, g)); // output hidden state (ht) let ht= builder. mul( o, builder. tanh( ct)); return [ ht, ct]; }
7.7.30. matmul
Compute the matrix product of two input tensors.partial interface MLGraphBuilder {MLOperand matmul (MLOperand a ,MLOperand b ,optional MLOperatorOptions options = {}); };partial dictionary MLOpSupportLimits {MLBinarySupportLimits matmul ; };
-
a
: anMLOperand
. The first input tensor which is at least 2-D. -
b
: anMLOperand
. The second input tensor which is at least 2-D. -
options
: anMLOperatorOptions
. Specifies the optional parameters of the operation.
Returns: an MLOperand
. The output tensor that contains the matrix
product of two input tensors.
-
If both a and b are 2-dimensional, they are multiplied like conventional matrices and produce a 2-dimensional tensor as the output.
-
If either a or b is
N
-dimensional whereN > 2
, it is treated as a stack of matrices with dimensions corresponding to the last two indices. The matrix multiplication will be broadcast according to [numpy-broadcasting-rule]. The shapes of a and b, except the last two dimensions, must be bidirectionally broadcastable. The output is aN
-dimensional tensor whose rank is the maximum rank of the input tensors. For each dimension, except the last two, of the output tensor, its size is the maximum size along that dimension of the input tensors.
MLOpSupportLimits
has following member for matmul()
:
matmul
, of type MLBinarySupportLimits-
Support limits for operator
matmul()
.
To calculate matmul output sizes, given MLOperand
a and MLOperand
b run the following steps:
-
Let rankA be a’s rank.
-
Let rankB be b’s rank.
-
If either rankA or rankB is less than 2, then throw a
TypeError
. -
Let colsA be shapeA[rankA - 1].
-
Let rowsA be shapeA[rankA - 2].
-
Let colsB be shapeB[rankB - 1].
-
Let rowsB be shapeB[rankB - 2].
-
Let batchShapeA be a clone of shapeA with the spatial dimensions (last 2 items) removed.
-
Let batchShapeB be a clone of shapeB with the spatial dimensions (last 2 items) removed.
-
Let outputShape be the result of bidirectionally broadcasting batchShapeA and batchShapeB. If that returns failure, then throw a
TypeError
. -
Append « rowsA, colsB » to outputShape.
-
Return outputShape.
The matmul(a, b, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and any of a and b returns false, then throw a
TypeError
. -
If a’s dataType is not
"float32"
or"float16"
, then throw aTypeError
. -
If b’s dataType is not equal to a’s dataType, then throw a
TypeError
. -
Let outputShape be the result of calculating matmul output sizes given a and b.
-
If that throws an error, re-throw the error.
-
-
Let desc be the result of creating an MLOperandDescriptor given a’s dataType and outputShape.
-
Make graph connections:
-
Let output be the result of creating an MLOperand given this and desc.
-
Let operator be an operator for the "matmul" operation, given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s inputs to a and b.
-
Set operator’s output to output.
-
-
Return output.
7.7.31. pad
Inflate the tensor with constant or mirrored values on the edges.enum {
MLPaddingMode ,
"constant" ,
"edge" ,
"reflection" };
"symmetric" dictionary :
MLPadOptions MLOperatorOptions {MLPaddingMode mode = "constant";MLNumber value = 0; };partial interface MLGraphBuilder {MLOperand pad (MLOperand input ,sequence <[EnforceRange ]unsigned long >beginningPadding ,sequence <[EnforceRange ]unsigned long >endingPadding ,optional MLPadOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits pad ; };
MLPadOptions
has the following members:
mode
, of type MLPaddingMode, defaulting to"constant"
-
The different ways to pad the tensor.
value
, of type MLNumber, defaulting to0
-
The padding value when
mode
is set to"constant"
.
-
input
: anMLOperand
. The input tensor. -
beginningPadding
: sequence<unsigned long
>. The number of padding values to add at the beginning of each input dimension, of length N where N is the rank of the input tensor. For each dimension d of input, beginningPadding[d] indicates how many values to add before the content in that dimension. -
endingPadding
: sequence<unsigned long
>. The number of padding values to add at the ending of each input dimension, of length N where N is the rank of the input tensor. For each dimension d of input, endingPadding[d] indicates how many values to add after the content in that dimension. -
options
: an optionalMLPadOptions
. The optional parameters of the operation.
Returns: an MLOperand
. The padded output tensor. Each dimension of the output tensor can be calculated as follows:
output size = beginning padding + input size + ending padding
MLOpSupportLimits
has following member for pad()
:
pad
, of type MLSingleInputSupportLimits-
Support limits for operator
pad()
.
To calculate padding output sizes, given input, beginningPadding and endingPadding, run the following steps:
The pad(input, beginningPadding, endingPadding, options)
method steps are:
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
If beginningPadding’s size and endingPadding’s size are not both equal to input’s rank, then throw a
TypeError
. -
Let desc be a copy of input.
[[descriptor]]
. -
Let outputShape be the result of calculating padding output sizes given input, beginningPadding and endingPadding.
-
If any item in outputShape is not a valid dimension, then throw a
TypeError
. -
Set options.
value
to the result of casting options.value
to input’s dataType. -
Set desc.
shape
to outputShape. -
Make graph connections:
-
Let output be the result of creating an MLOperand given this and desc.
-
Let operator be an operator for the "padding" operation, given beginningPadding, endingPadding and options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
Examples for constant, edge, reflection and symmetric padding:
// input: [[1,2,3], [4,5,6]] const input= builder. constant( { dataType: 'float32' , shape: [ 2 , 3 ]}, new Float32Array([ 1 , 2 , 3 , 4 , 5 , 6 ])); const beginningPadding= [ 1 , 2 ]; const endingPadding= [ 1 , 2 ]; // "constant" padded: // [[0,0,0,0,0,0,0], // [0,0,1,2,3,0,0], // [0,0,4,5,6,0,0], // [0,0,0,0,0,0,0]] builder. pad( input, beginningPadding, endingPadding); // "edge" padded: // [[1,1,1,2,3,3,3], // [1,1,1,2,3,3,3], // [4,4,4,5,6,6,6], // [4,4,4,5,6,6,6]] builder. pad( input, beginningPadding, endingPadding, { mode: 'edge' }); // "reflection" padded: // [[6,5,4,5,6,5,4], // [3,2,1,2,3,2,1], // [6,5,4,5,6,5,4], // [3,2,1,2,3,2,1]] builder. pad( input, beginningPadding, endingPadding, { mode: 'reflection' }); // "symmetric" padded: // [[2,1,1,2,3,3,2], // [2,1,1,2,3,3,2], // [5,4,4,5,6,6,5], // [5,4,4,5,6,6,5]] builder. pad( input, beginningPadding, endingPadding, { mode: 'symmetric' });
7.7.32. Pooling operations
Compute a pooling operation across all the elements within the moving window over the input tensor.enum {
MLRoundingType ,
"floor" };
"ceil" dictionary :
MLPool2dOptions MLOperatorOptions {sequence <[EnforceRange ]unsigned long >windowDimensions ;sequence <[EnforceRange ]unsigned long >padding ;sequence <[EnforceRange ]unsigned long >strides ;sequence <[EnforceRange ]unsigned long >dilations ;MLInputOperandLayout layout = "nchw";MLRoundingType roundingType = "floor";sequence <[EnforceRange ]unsigned long >outputSizes ; };partial interface MLGraphBuilder {MLOperand averagePool2d (MLOperand input ,optional MLPool2dOptions options = {});MLOperand l2Pool2d (MLOperand input ,optional MLPool2dOptions options = {});MLOperand maxPool2d (MLOperand input ,optional MLPool2dOptions options = {}); };partial dictionary MLOpSupportLimits {MLSingleInputSupportLimits averagePool2d ;MLSingleInputSupportLimits l2Pool2d ;MLSingleInputSupportLimits maxPool2d ; };
MLPool2dOptions
has the following members:
windowDimensions
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 2: [windowHeight, windowWidth]. Specifies the dimensions of the sliding window. The default value for the window dimensions are the height and width dimensions of the input shape.
padding
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 4: [beginningHeight, endingHeight, beginningWidth, endingWidth]. Specifies the additional rows and columns added to the beginning and ending of each spatial dimension of the convolution input. The default value is [0,0,0,0].
strides
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 2: [strideHeight, strideWidth]. Specifies the stride of the sliding window for each spatial dimension of the convolution input. The default value is [1,1].
dilations
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 2: [dilationHeight, dilationWidth]. Specifies the dilation factor for each spatial dimension applied on the convolution filter (kernel). The default value is [1,1].
layout
, of type MLInputOperandLayout, defaulting to"nchw"
-
Specifies the layout format of the input and output tensor as follows:
roundingType
, of type MLRoundingType, defaulting to"floor"
-
The rounding function used to compute the output shape.
outputSizes
, of typesequence<[EnforceRange] unsigned long>
-
A list of length 2. Specifies the sizes of the two spacial dimensions of the output tensor. When the output sizes are explicitly specified, the
roundingType
is ignored.If not specified, the output sizes are automatically computed.
-
input
: anMLOperand
. The input 4-D tensor. The logical shape is interpreted according to the value of options.layout. -
options
: an optionalMLPool2dOptions
. The optional parameters of the operation.
Returns: an MLOperand
. The output 4-D tensor that contains the
result of the reduction. The logical shape is interpreted according to the
value of layout. More specifically, if the options.roundingType is "floor"
, the spatial dimensions of the output tensor can be calculated as follows:
output size = floor(1 + (input size - filter size + beginning padding + ending padding) / stride)
or if options.roundingType is "ceil"
:
output size = ceil(1 + (input size - filter size + beginning padding + ending padding) / stride)
MLOpSupportLimits
has following members for pooling operations:
averagePool2d
, of type MLSingleInputSupportLimits-
Support limits for operator
averagePool2d()
. l2Pool2d
, of type MLSingleInputSupportLimits-
Support limits for operator
l2Pool2d()
. maxPool2d
, of type MLSingleInputSupportLimits-
Support limits for operator
maxPool2d()
.
// 'global' max pooling builder. maxPool2d( input);
To calculate pool2d output sizes given MLInputOperandLayout
layout, list of 4 unsigned integers inputShape, MLRoundingType
roundingType, list of 2 unsigned integers windowDimensions, list of 4 unsigned integers padding, list of 2 unsigned integers strides, list of 2 unsigned integers dilations, and optional list of 2 unsigned integers outputSizes, perform these steps. They return a list of 4 unsigned integers.
-
Switch on layout:
-
If outputSizes is not given, then:
-
Let outputHeight be outputSizes[0].
-
Let outputWidth be outputSizes[1].
-
-
Otherwise:
-
Let outputSizes be the result of calculating conv2d output sizes given inputHeight, inputWidth, windowDimensions[0], windowDimensions[1], padding, strides, and dilations.
-
Let outputHeight be outputSizes[0].
-
Let outputWidth be outputSizes[1].
-
Switch on roundingType:
-
-
Switch on layout:
To create pooling operation given string op, MLOperand
input, MLPool2dOptions
options, and optional list allowedDataTypes, run the following steps:
-
Assert: op is one of "averagePool2d", "l2Pool2d", "maxPool2d".
-
If this.
[[hasBuilt]]
is true, then throw an "InvalidStateError
"DOMException
. -
If validating operand with this and input returns false, then throw a
TypeError
. -
If allowedDataTypes is given and it does not contain input’s dataType, then throw a
TypeError
. -
If options.
windowDimensions
exists and its size is not 2, then throw aTypeError
. -
Otherwise, set options.
windowDimensions
to the height and width dimensions of the shape of input. -
If options.
outputSizes
exists, or if options.padding
does not exist, set options.padding
to the list « 0, 0, 0, 0 ». -
If options.
strides
does not exist, set options.strides
to the list « 1, 1 ». -
If any value in options.
strides
is not greater than 0, then throw aTypeError
. -
If options.
outputSizes
exists: -
If options.
dilations
does not exist, set options.dilations
to the list « 1, 1 ». -
If options.
dilations
's size is not 2, then throw aTypeError
. -
If any value in options.
dilations
is not greater than 0, then throw aTypeError
. -
Let desc be a copy of input.
[[descriptor]]
. -
Let outputShape be the result of calculating pool2d output sizes given options.
layout
, input’s shape, options.roundingType
, options.windowDimensions
, options.padding
, options.strides
, options.dilations
, and options.outputSizes
(if it exists). -
If any item in outputShape is not a valid dimension, then throw a
TypeError
. -
Set desc.
shape
to outputShape. -
Make graph connections:
-
Let output be the result of creating an MLOperand given this and desc.
-
Let operator be an operator for the op operation, given options.
-
Set output.
[[operator]]
to operator. -
Set operator’s input to input.
-
Set operator’s output to output.
-
-
Return output.
The following pooling algorithms are supported.
averagePool2d(input, options)
method steps are:
-
Let output be the result of running the create pooling operation given "averagePool2d", input, options, and «
"float32"
,"float16"
». -
Return output.
l2Pool2d(input, options)
method steps are:
-
Let output be the result of running the create pooling operation given "l2Pool2d", input, options, and «
"float32"
,"float16"
». -
Return output.
maxPool2d(input, options)
method steps are:
-
Let output be the result of running the create pooling operation given "maxPool2d", input and options.
-
Return output.
7.7.32.1. averagePool2d
Calculate the average value for patches of a feature map, and use it to create a pooled feature map. See § 7.7.32 Pooling operations for more detail.7.7.32.2. l2Pool2d
Apply the L2 norm function to a region of the input feature map. The L2 norm is the square root of the sum of the squares of its elements. See § 7.7.32 Pooling operations for more detail.7.7.32.3. maxPool2d
Calculate the maximum value for patches of a feature map, and use it to create a pooled feature map. See § 7.7.32 Pooling operations for more detail.7.7.33. prelu
Calculate the parametric version of rectified linear function (Parametric ReLU) on the input tensor element-wise. Parametric ReLU is a type of leaky ReLU that, instead of having a scalar slope like 0.01, making the slope (coefficient of leakage) into a parameter that is learned during the model training phase of this operation. The calculation follows the expressionmax(0, x) + slope * min(0, x)
.
The operation will be broadcast according to [numpy-broadcasting-rule]. The input tensors must be bidirectionally broadcastable. The rank of the output tensor is the maximum rank of the input tensors. For each dimension of the output tensor, its size is the maximum size along that dimension of the input tensors.
partial interface MLGraphBuilder {MLOperand prelu (MLOperand input ,MLOperand slope ,optional MLOperatorOptions options = {}); };dictionary {
MLPreluSupportLimits MLSupportLimits input ;MLSupportLimits slope ;MLSupportLimits output ; };partial dictionary MLOpSupportLimits {MLPreluSupportLimits prelu ; };