Arsitektur Sekuensial Lanjut: CNN-1D, Seq2Seq, & Attention
Temporal Convolutional Networks, Encoder-Decoder, dan Pengenalan Transformer Deret Waktu
Sub-CPMK 11: Mampu merancang 1D-CNN temporal, arsitektur Seq2Seq, dan memahami mekanisme Self-Attention Transformer.
Operasi konvolusi 1D temporal, Causal & Dilated Convolutions untuk memperluas Receptive Field, arsitektur Encoder-Decoder (Seq2Seq) untuk peramalan multi-cakrawala, mekanisme Bahdanau dan Luong Attention, serta paradigma Transformer (PatchTST / Informer).
Definisi Causal Convolutions dan formula Scaled Dot-Product Attention.
Perhitungan Effective Receptive Field (ERF) pada TCN 3-layer dan analisis matriks visualisasi Attention Weights.
Evaluasi efisiensi inferensi real-time model Transformer vs Autoregressive RNN dan perancangan CNN-BiLSTM-Attention.
import torch
import torch.nn as nn
import torch.nn.functional as F
class ScaledDotProductAttention(nn.Module):
def __init__(self, d_k):
super().__init__()
self.d_k = d_k
def forward(self, Q, K, V):
scores = torch.matmul(Q, K.transpose(-2, -1)) / np.sqrt(self.d_k)
attn_weights = F.softmax(scores, dim=-1)
return torch.matmul(attn_weights, V), attn_weights