OmegaPrep

We don’t break die
 

The OmegaPrep was designed to perform in package die thinning and polishing in addition to a  number of machining functions for sample preparation. The die thinned and polished by the OmegaPrep are compatible with all backside procedures such as IR emission microscopy and FIB edit.  OmegaPrep can also perform mechanical decapsulation, removal of heat spreaders and heat sinks, and can easily remove  plastic packaging and die attach pad to expose the die. Additional capabilities have been included that provide processes for die de-layering &/or de-processing; substrate or PCB delayering; fixture machining; selected area thinning to 1µm remaining silicon, and more.

 

OmegaPrep is a specially designed, CNC machining system controlled by a graphic user interface that allows for the operator to select the desired functions and enter the key device measurements and process parameters. There is no need to know anything about machining or “G Code” to operate the OmegaPrep. There are no knobs to adjust, or cranks to turn. ‘Speeds and Feeds’ are  preset for each process to  optimize processing  results.

 

Automatic tool touch-down detection, video alignment and special profiling functions make set-device set up simple and easy. The surface profiling function eliminates the need for die leveling and maintains a constant resulting thickness for both die thinning and consistent layer removal during  delayering processes.

 

Die as large as 44mm (square) and delayering and machining areas up to 55mm square can be accommodated. Packages up to 150mm long x 125mm wide can be mounted on the standard fixtures supplied with the system.  Custom fixtures can be supplied for even larger devices.

At the beginning of each process, the OmegaPrep will measure the surface profile of the sample. This is done by contacting the surface at discrete points and recording the surface height at each measured point. The touch tool used has a spherical tip to eliminate surface damage from such contacts. The measured profile points are used to generate a ‘wire frame model’ representation of the surface that is  subsequently used to generate the three dimensional path of the tool tip during process. The measurement of the surface profile completely eliminates the need for any sample leveling adjustments. The number of measured points and their positions are selected by the OmegaPrep on the basis of the sample size and the process to be run. For very small samples, only four points are measured. The four points are used to define the average plane of the surface and is all that is required as the variation in surface profile from a plane is limited by the mechanical properties of the sample.

 

Between the process steps during  die thinning, the sample holder and sample are removed for cleaning. Often, due to possible particulates trapped between the sample holder and the stage, the sample profile may be rotated slightly from the initial profile measurements. This is corrected for by measuring a small number of points on the surface and mathematically rotating the stored profile to match.  Some processes allow for the operator to input silicon thickness measurements. The optional thickness measurement systems can be used to measure the remaining silicon thickness. The thickness profile, either entered by the operator, or measured by the OmegaPrep, will be used to correct the mechanical surface profile to maintain a constant silicon thickness over the area being processed.

 

Once the device and process parameters, die size, die position, die attach pad size, amount of removal, etc., have been programmed, they can be saved as a process recipe. The recipe can be recalled for reference, or duplication of the processing for a different sample. Up to 96 recipes can be stored in the OmegaPrep, each with a unique, operator generated name.

 

An in-depth look at Google’s first Tensor Processing Unit (TPU) 
 

There’s a common thread that connects Google services such as Search, Street View, Photos and Translate: they all use Google’s Tensor Processing Unit, or TPU, to accelerate their neural network computations behind the scenes.

 

The TPU ASIC is built on a 28nm process, runs at 700MHz and consumes 40W when running. Because we needed to deploy the TPU to Google’s existing servers as fast as possible, we chose to package the processor as an external accelerator card that Nts into an SATA hard disk slot for drop-in installation. The TPU is connected to its host via a PCIe Gen3 x16 bus that provides 12.5GB/s of effective bandwidth.

 

In July 2016, a survey of six representative neural network applications across Google’s production services and summed up the total number of weights in each neural network architecture. You can see the results in the table below.

Type of network	# of network layers	# of weights	% of deployed
MLP0	5	20M	61%
MLP1	4	5M
LSTM0	58	52M	29%
LSTM1	56	34M
CNN0	16	8M	5%
CNN1	89	100M

Neural network predictions often don’t require the precision of floating point calculations with 32-bit or even 16-bit numbers. With some effort, you may be able to use 8-bit integers to calculate a neural network prediction and still maintain the appropriate level of accuracy.

 

Quantization is an optimization technique that uses an 8-bit integer to approximate an arbitrary value between a preset minimum and a maximum value.

 

Quantization is a powerful tool for reducing the cost of neural network predictions, and the corresponding reductions in memory usage are important as well, especially for mobile and embedded deployments. For example, when you apply quantization to Inception, the popular image recognition model, it gets compressed from 91MB to 23MB, about one-fourth the original size.

 

Being able to use integer rather than ooating point operations greatly reduces the hardware footprint and energy consumption of our TPU. A TPU contains 65,536 8-bit integer multipliers. The popular GPUs used widely on the cloud environment contains a few thousands of 32-bit ooating-point multipliers. As long as you can meet the accuracy requirements of your application with 8­bits, that can be up to 25X or more multipliers.

Most modern CPUs are heavily inouenced by the Reduced Instruction Set Computer (RISC) design style. With RISC, the focus is to deNne simple instructions (e.g., load, store, add and multiply) that are commonly used by the majority of applications and then to execute those instructions as fast as possible. The Complex Instruction Set Computer (CISC) style as the basis of the TPU instruction set instead. A CISC design focuses on implementing high-level instructions that run more complex tasks (such as calculating multiply-and-add many times) with each instruction. Let’s take a look at the block diagram of the TPU.

• Matrix Multiplier Unit (MXU): 65,536 8-bit multiply-and-add units for matrix operations
• Unified Buffer (UB): 24MB of SRAM that work as registers
• Activation Unit (AU): Hardwired activation functions

 

This instruction set focuses on the major mathematical operations required for neural network inference that we mentioned earlier: execute a matrix multiply between input data and weights and apply an activation function.

In short, the TPU design encapsulates the essence of neural network calculation, and can be programmed for a wide variety of neural network models. To program it, we created a compiler and software stack that translates API calls from TensorFlow graphs into TPU instructions.



Parallel Processing on the Matrix Multiplier Unit
 

Typical RISC processors provide instructions for simple calculations such as multiplying or adding numbers. These are so-called scalar processors, as they process a single operation (= scalar operation) with each instruction.

 

Even though CPUs run at clock speeds in the gigahertz range, it can still take a long time to execute large matrix operations via a sequence of scalar operations. One effective and well-known way to improve the performance of such large matrix operations is through vector processing, where the same operation is performed concurrently across a large number of data elements at the same time. CPUs incorporate instruction set extensions such as SSE and AVX that express such vector operations. The streaming multiprocessors (SMs) of GPUs are effectively vector processors, with many such SMs on a single GPU die. Machines with vector processing support can process hundreds to thousands of operations in a single clock cycle.



In the case of the TPU, Google designed its MXU as a matrix processor that processes hundreds of thousands of operations (= matrix operation) in a single clock cycle. Think of it like printing documents one character at a time, one line at a time and a page at a time.

For an MXU, however, matrix multiplication reuses both inputs many times as part of producing the output. We can read each input value once, but use it for many different operations without storing it back to a register. Wires only connect spatially adjacent ALUs, which makes them short and energy-eucient. The ALUs perform only multiplications and additions in Nxed patterns, which simpliNes their design.

 

The design is called systolic because the data oows through the chip in waves, reminiscent of the way that the heart pumps blood. The particular kind of systolic array in the MXU is optimized for power and area euciency in performing matrix multiplications, and is not well suited for general-purpose computation. It makes an engineering tradeoff: limiting registers, control and operational oexibility in exchange for euciency and much higher operation density.

The TPU Matrix Multiplication Unit has a systolic array mechanism that contains 256 • 256 = total 65,536 ALUs. That means a TPU can process 65,536 multiply-and-adds for 8-bit integers every cycle. Because a TPU runs at 700MHz, a TPU can compute 65,536 • 700,000,000 = 46 • 1012  multiply-and-add operations or 92 Teraops per second (92 • 10 12) in the matrix unit.

	Operations per cycle
CPU	a few
CPU (vector extension)	tens
GPU	tens of thousands
TPU	hundreds of thousands, up to 128K

Minimal and deterministic design
 

Another signiNcant beneNt of designing a new processor optimized for neural network inference is that you can be the ultimate minimalist in your design. As stated in our TPU paper:

Because general-purpose processors such as CPUs and GPUs must provide good performance across a wide range of applications, they have evolved myriad sophisticated, performance­oriented mechanisms. As a side effect, the behavior of those processors can be diucult to predict, which makes it hard to guarantee a certain latency limit on neural network inference. In contrast, TPU design is strictly minimal and deterministic as it has to run only one task at a time: neural network prediction. You can see its simplicity in the ooor plan of the TPU die.

Neural network predictions support end-user-facing products and services, and everyone knows that users become impatient if a service takes too long to respond. Thus, for the MLP0 application, we limit the 99th-percentile prediction latency to around 7 ms, for a consistently fast user experience from TPU-based Google services. The following is an overall performance (predictions per second) comparison between the TPU and a contemporary CPU and GPU across six neural network applications under a latency limit. In the most spectacular case, the TPU provides 71X performance compared with the CPU for the CNN1 application.


Neural-specific architecture
 

In this article, we’ve seen that the secret of TPU’s outstanding performance is its dedication to neural network inference. The quantization choices, CISC instruction set, matrix processor and minimal design all became possible when we decided to focus on neural network inference. Google felt conNdent investing in the TPU because we see that neural networks are driving a paradigm shift in computing and we expect TPUs to be an essential part of fast, smart and affordable services for years to come.