To define our network, we should succeed class nn.Module and implement the function forward. We put all the layers we want in the function __init__() and define how layers connect in function forward.

Example

class Hopenet(nn.Module):
# Hopenet with 3 output layers for yaw, pitch and roll
# Predicts Euler angles by binning and regression with the expected value
def __init__(self, block, layers, num_bins):
self.inplanes = 64
super(Hopenet, self).__init__()
self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3,
bias=False)
self.bn1 = nn.BatchNorm2d(64)
self.relu = nn.ReLU(inplace=True)
self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
self.layer1 = self._make_layer(block, 64, layers[0])
self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
self.avgpool = nn.AvgPool2d(7)
self.fc_yaw = nn.Linear(512 * block.expansion, num_bins)
self.fc_pitch = nn.Linear(512 * block.expansion, num_bins)
self.fc_roll = nn.Linear(512 * block.expansion, num_bins)
# Vestigial layer from previous experiments
self.fc_finetune = nn.Linear(512 * block.expansion + 3, 3)
for m in self.modules():
if isinstance(m, nn.Conv2d):
n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
m.weight.data.normal_(0, math.sqrt(2. / n))
elif isinstance(m, nn.BatchNorm2d):
m.weight.data.fill_(1)
m.bias.data.zero_()

1.Constructor

super(Hopenet, self).__init__(): call the construtor in superclass

2.Layers

nn.Conv2d: class torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)

nn.BatchNorm2d: class torch.nn.BatchNorm2d(num_features, eps=1e-05, momentum=0.1, affine=True) For each channel (the second dimension), the batch normalization computes the mean and variance once.

nn.ReLU: class torch.nn.ReLU(inplace=False) The inplace will cover the old value with a new value and save the memory.

nn.MaxPool2d: class torch.nn.MaxPool2d(kernel_size, stride=None, padding=0, dilation=1, return_indices=False, ceil_mode=False)

nn.AvgPool2d: class torch.nn.AvgPool2d(kernel_size, stride=None, padding=0, ceil_mode=False, count_include_pad=True)

nn.Linear: class torch.nn.Linear(in_features, out_features, bias=True)

3.Initialization

for m in self.modules():
if isinstance(m, nn.Conv2d):
n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
m.weight.data.normal_(0, math.sqrt(2. / n))
elif isinstance(m, nn.BatchNorm2d):
m.weight.data.fill_(1)
m.bias.data.zero_()

The kernel is the weight and the gamma and beta are the weight and bias.

You can see that only the first convolutional layer in every stage has a residual layer.

The layer is calculated as follows: the first convolutional layer with 7x7 kernel, the max pool layer and the next 48 blocks add up to 50. The network is like:

This error occurs because of BGSAVE being failed. A lot of the times BGSAVE fails because the fork can't allocate memory. Many times the fork fails to allocate memory (although the machine has enough RAM available) because of a conflicting optimization by the OS.