tensor

Posted on Edited on In

Definition

Tensor--"张量", actually it is a little bit like the vector to store numbers(linear algebra)

Operation

import torch

第一步永远是先导入torch库才能使用

0 Initializatin

  1. x = torch.empty(a, b) a rows and b columns of 0
  2. x = torch.random(a, b) a rows and b columns of random number
  3. x = torch.zeros(a, b, dtype = torch.long) use dtype to specify the type of the entries in tensor(in this case, type is long)
  4. x = torch.tensor([x1, x2, ...]) use square brackets to init the size/ values of a tensor
  5. y = torch.random_like(x, detype = torch.float) , _likemesans using an existing tensor to init another one(in this case, we use the size of the old tensor to create a new tensor with type float)

More functions to init

函数 功能
Tensor(*sizes) 基础构造函数
tensor(data,) 类似np.array的构造函数
ones(*sizes) 全1Tensor
zeros(*sizes) 全0Tensor
eye(*sizes) 对角线为1，其他为0
arange(s,e,step) 从s到e，步长为step
linspace(s,e,steps) 从s到e，均匀切分成steps份
rand/randn(*sizes) 均匀/标准分布
normal(mean,std)/uniform(from,to) 正态分布/均匀分布
randperm(m) 随机排列

1 Addition

The following 4 ways are same with same size tensors x y z 1. z = x + y 2. z = torch.add(x, y) 3. torch.add(x, y, out = z) 4. y.add_(x) note: this is an inplace function, each inplace function has a postfix_

2 Index

1
2
y  = x[0,: ] # y points to all of x
y += 1 # all entries will plus 1

note: with this method, y will share memory with x, so you cannot only modify y without touching x

More high level function to select index

函数 功能
index_select(input, dim, index) 在指定维度dim上选取，比如选取某些行、某些列
masked_select(input, mask) 如上，a[a>0]，使用ByteTensor进行选取
nonzero(input) 非0元素的下标
gather(input, dim, index) 根据index，在dim维度上选取数据，输出的size与index一样

3 Shape changing

  1. use view()function

    y = x.view(a*b)change the shape of x to 1 row 1*b column

    x.size() = torch.size([a,b]), y.size() = torch.size([a*b])

    note: view() function still shares memory and does not change the number of entries

    if you want a copy just use clone

    eg. x_copy = x.clone().veiw(new_a, new_b) ### more item() function:

--change a tensor to a number

1
2
x = torch.tensor([x1])
y = x.item() # y will be a number = x1

Note: it only happenes when there is only one 1 entry in tensor x

some linear operation

函数 功能
trace 对角线元素之和(矩阵的迹)
diag 对角线元素
triu/tril 矩阵的上三角/下三角，可指定偏移量
mm/bmm 矩阵乘法，batch的矩阵乘法
addmm/addbmm/addmv/addr/baddbmm.. 矩阵运算
t 转置
dot/cross 内积/外积
inverse 求逆矩阵
svd 奇异值分解

3 Broadcasting

Operations on two tensors with different shape generally, tensors will be broadcast to larger size that can fit the operation.

--One tensor's dimension must be one

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
tensor x is 3*5, then y must be 1*5 or 3*1
eg.
x = torch.arange(0, 15).view(3, 5)
y = torch.arange(0, 5).view(1, 5)
#x + y = tensor([[ 0,  2,  4,  6,  8],
                 [ 5,  7,  9, 11, 13],
                 [10, 12, 14, 16, 18]])
y = torch.arange(0,3).view(3,1)
#x + y = tensor([[ 0,  1,  2,  3,  4],
                 [ 6,  7,  8,  9, 10],
                 [12, 13, 14, 15, 16]])
more: 
x = torch.arange(0,5).view(1,5)
y = torch.arange(0,3).view(3,1)
#x + y = tensor([[0, 1, 2, 3, 4],
                 [1, 2, 3, 4, 5],
                 [2, 3, 4, 5, 6]])

Memory Overhead

  1. there is no memory overhead using index, just copy the address instead of creatign a new memory space.
  2. mathematical operations will create a new memory space.
  3. torch.add(x, y, out=y) && y += x(y.add_(x)) also create new space

! Specaial note: although view() function will create a tensor sharing data with the old one(you can modify one to modify both), but new tensor has a new memory, because view() function just share date and tensor has more attributes.

Exchange between tensor and numpy(数组)

import numpy as np always remember to import

Using functions:

numpy() #chang tensor to numpy

from_numpy() #change numpy to tensor

Note:

  1. created numpy/tensor also shares memory(modify one to modify both), so changing between tensor and numpy is really fast
  2. only tensor(containing numbers) can be changed to numpy, chartensor(containing characters) cannot

More: If you do not want to share memory, just use torch.tensor(numpy)

Tensor on GPU

1
2
3
device = torch.device("cuda") #specify device to GPU
x = x.to(device) #transfer tensor to GPU
#or just x.to("cuda")
1
y = torch.ones(a, b, device=device) #init a tensor in GPU