Merge FPGA repository back into UHD repository
The FPGA codebase was removed from the UHD repository in 2014 to reduce the size of the repository. However, over the last half-decade, the split between the repositories has proven more burdensome than it has been helpful. By merging the FPGA code back, it will be possible to create atomic commits that touch both FPGA and UHD codebases. Continuous integration testing is also simplified by merging the repositories, because it was previously difficult to automatically derive the correct UHD branch when testing a feature branch on the FPGA repository. This commit also updates the license files and paths therein. We are therefore merging the repositories again. Future development for FPGA code will happen in the same repository as the UHD host code and MPM code. == Original Codebase and Rebasing == The original FPGA repository will be hosted for the foreseeable future at its original local location: https://github.com/EttusResearch/fpga/ It can be used for bisecting, reference, and a more detailed history. The final commit from said repository to be merged here is 05003794e2da61cabf64dd278c45685a7abad7ec. This commit is tagged as v4.0.0.0-pre-uhd-merge. If you have changes in the FPGA repository that you want to rebase onto the UHD repository, simply run the following commands: - Create a directory to store patches (this should be an empty directory): mkdir ~/patches - Now make sure that your FPGA codebase is based on the same state as the code that was merged: cd src/fpga # Or wherever your FPGA code is stored git rebase v4.0.0.0-pre-uhd-merge Note: The rebase command may look slightly different depending on what exactly you're trying to rebase. - Create a patch set for your changes versus v4.0.0.0-pre-uhd-merge: git format-patch v4.0.0.0-pre-uhd-merge -o ~/patches Note: Make sure that only patches are stored in your output directory. It should otherwise be empty. Make sure that you picked the correct range of commits, and only commits you wanted to rebase were exported as patch files. - Go to the UHD repository and apply the patches: cd src/uhd # Or wherever your UHD repository is stored git am --directory fpga ~/patches/* rm -rf ~/patches # This is for cleanup == Contributors == The following people have contributed mainly to these files (this list is not complete): Co-authored-by: Alex Williams <alex.williams@ni.com> Co-authored-by: Andrej Rode <andrej.rode@ettus.com> Co-authored-by: Ashish Chaudhari <ashish@ettus.com> Co-authored-by: Ben Hilburn <ben.hilburn@ettus.com> Co-authored-by: Ciro Nishiguchi <ciro.nishiguchi@ni.com> Co-authored-by: Daniel Jepson <daniel.jepson@ni.com> Co-authored-by: Derek Kozel <derek.kozel@ettus.com> Co-authored-by: EJ Kreinar <ej@he360.com> Co-authored-by: Humberto Jimenez <humberto.jimenez@ni.com> Co-authored-by: Ian Buckley <ian.buckley@gmail.com> Co-authored-by: Jörg Hofrichter <joerg.hofrichter@ni.com> Co-authored-by: Jon Kiser <jon.kiser@ni.com> Co-authored-by: Josh Blum <josh@joshknows.com> Co-authored-by: Jonathon Pendlum <jonathan.pendlum@ettus.com> Co-authored-by: Martin Braun <martin.braun@ettus.com> Co-authored-by: Matt Ettus <matt@ettus.com> Co-authored-by: Michael West <michael.west@ettus.com> Co-authored-by: Moritz Fischer <moritz.fischer@ettus.com> Co-authored-by: Nick Foster <nick@ettus.com> Co-authored-by: Nicolas Cuervo <nicolas.cuervo@ettus.com> Co-authored-by: Paul Butler <paul.butler@ni.com> Co-authored-by: Paul David <paul.david@ettus.com> Co-authored-by: Ryan Marlow <ryan.marlow@ettus.com> Co-authored-by: Sugandha Gupta <sugandha.gupta@ettus.com> Co-authored-by: Sylvain Munaut <tnt@246tNt.com> Co-authored-by: Trung Tran <trung.tran@ettus.com> Co-authored-by: Vidush Vishwanath <vidush.vishwanath@ettus.com> Co-authored-by: Wade Fife <wade.fife@ettus.com> Original-commit: bafa9d95453387814ef25e6b6256ba8db2df612f
This commit is contained in:
Martin Braun
co-authored by
Alex Williams
Andrej Rode
Ashish Chaudhari
Ben Hilburn
Ciro Nishiguchi
Daniel Jepson
Derek Kozel
EJ Kreinar
Humberto Jimenez
Ian Buckley
Jörg Hofrichter
Jon Kiser
Josh Blum
Jonathon Pendlum
Matt Ettus
Michael West
Moritz Fischer
Nick Foster
Nicolas Cuervo
Paul Butler
Paul David
Ryan Marlow
Sugandha Gupta
Sylvain Munaut
Trung Tran
Vidush Vishwanath
Wade Fife
Alex Williams
Andrej Rode
Ashish Chaudhari
Ben Hilburn
Ciro Nishiguchi
Daniel Jepson
Derek Kozel
EJ Kreinar
Humberto Jimenez
Ian Buckley
Jörg Hofrichter
Jon Kiser
Josh Blum
Jonathon Pendlum
Matt Ettus
Michael West
Moritz Fischer
Nick Foster
Nicolas Cuervo
Paul Butler
Paul David
Ryan Marlow
Sugandha Gupta
Sylvain Munaut
Trung Tran
Vidush Vishwanath
Wade Fife
parent
74893643ca
commit
6b67702ad7
@@ -0,0 +1,52 @@
|
||||
#
|
||||
# Copyright 2015 Ettus Research LLC
|
||||
#
|
||||
|
||||
#-------------------------------------------------
|
||||
# Top-of-Makefile
|
||||
#-------------------------------------------------
|
||||
# Define BASE_DIR to point to the "top" dir
|
||||
BASE_DIR = $(abspath ../../../../top)
|
||||
# Include viv_sim_preamble after defining BASE_DIR
|
||||
include $(BASE_DIR)/../tools/make/viv_sim_preamble.mak
|
||||
|
||||
#-------------------------------------------------
|
||||
# Design Specific
|
||||
#-------------------------------------------------
|
||||
# Define part using PART_ID (<device>/<package>/<speedgrade>)
|
||||
ARCH = kintex7
|
||||
PART_ID = xc7k410t/ffg900/-2
|
||||
|
||||
# Include makefiles and sources for the DUT and its dependencies
|
||||
include $(BASE_DIR)/../lib/control/Makefile.srcs
|
||||
include $(BASE_DIR)/../lib/fifo/Makefile.srcs
|
||||
include $(BASE_DIR)/../lib/rfnoc/crossbar/Makefile.srcs
|
||||
include $(BASE_DIR)/../lib/rfnoc/core/Makefile.srcs
|
||||
|
||||
DESIGN_SRCS = $(abspath \
|
||||
$(FIFO_SRCS) \
|
||||
$(CONTROL_LIB_SRCS) \
|
||||
$(RFNOC_XBAR_SRCS) \
|
||||
$(RFNOC_CORE_SRCS) \
|
||||
)
|
||||
|
||||
#-------------------------------------------------
|
||||
# Testbench Specific
|
||||
#-------------------------------------------------
|
||||
# Define only one toplevel module
|
||||
TB_TOP_MODULE ?= crossbar_tb
|
||||
SIM_TOP = $(TB_TOP_MODULE)
|
||||
|
||||
SIM_SRCS = \
|
||||
$(abspath chdr_traffic_source_sim.sv) \
|
||||
$(abspath chdr_traffic_sink_sim.sv) \
|
||||
$(abspath crossbar_tb.sv) \
|
||||
$(abspath $(TB_TOP_MODULE).sv)
|
||||
|
||||
#-------------------------------------------------
|
||||
# Bottom-of-Makefile
|
||||
#-------------------------------------------------
|
||||
# Include all simulator specific makefiles here
|
||||
# Each should define a unique target to simulate
|
||||
# e.g. xsim, vsim, etc and a common "clean" target
|
||||
include $(BASE_DIR)/../tools/make/viv_simulator.mak
|
||||
@@ -0,0 +1,51 @@
|
||||
#
|
||||
# Copyright 2015 Ettus Research LLC
|
||||
#
|
||||
|
||||
#-------------------------------------------------
|
||||
# Top-of-Makefile
|
||||
#-------------------------------------------------
|
||||
# Define BASE_DIR to point to the "top" dir
|
||||
BASE_DIR = $(abspath ../../../../../top)
|
||||
# Include viv_sim_preamble after defining BASE_DIR
|
||||
include $(BASE_DIR)/../tools/make/viv_sim_preamble.mak
|
||||
|
||||
#-------------------------------------------------
|
||||
# Design Specific
|
||||
#-------------------------------------------------
|
||||
# Define part using PART_ID (<device>/<package>/<speedgrade>)
|
||||
ARCH = kintex7
|
||||
PART_ID = xc7k410t/ffg900/-2
|
||||
|
||||
# Include makefiles and sources for the DUT and its dependencies
|
||||
include $(BASE_DIR)/../lib/control/Makefile.srcs
|
||||
include $(BASE_DIR)/../lib/fifo/Makefile.srcs
|
||||
include $(BASE_DIR)/../lib/rfnoc/crossbar/Makefile.srcs
|
||||
include $(BASE_DIR)/../lib/rfnoc/core/Makefile.srcs
|
||||
|
||||
DESIGN_SRCS = $(abspath \
|
||||
$(FIFO_SRCS) \
|
||||
$(CONTROL_LIB_SRCS) \
|
||||
$(RFNOC_XBAR_SRCS) \
|
||||
$(RFNOC_CORE_SRCS) \
|
||||
)
|
||||
|
||||
#-------------------------------------------------
|
||||
# Testbench Specific
|
||||
#-------------------------------------------------
|
||||
# Define only one toplevel module
|
||||
SIM_TOP = axis_ctrl_crossbar_nxn_tb
|
||||
|
||||
SIM_SRCS = \
|
||||
$(abspath axis_ctrl_crossbar_nxn_tb.sv) \
|
||||
$(abspath ../crossbar_tb.sv) \
|
||||
$(abspath ../chdr_traffic_source_sim.sv) \
|
||||
$(abspath ../chdr_traffic_sink_sim.sv)
|
||||
|
||||
#-------------------------------------------------
|
||||
# Bottom-of-Makefile
|
||||
#-------------------------------------------------
|
||||
# Include all simulator specific makefiles here
|
||||
# Each should define a unique target to simulate
|
||||
# e.g. xsim, vsim, etc and a common "clean" target
|
||||
include $(BASE_DIR)/../tools/make/viv_simulator.mak
|
||||
@@ -0,0 +1,26 @@
|
||||
//
|
||||
// Copyright 2018 Ettus Research, A National Instruments Company
|
||||
//
|
||||
// SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
|
||||
|
||||
`timescale 1ns/1ps
|
||||
|
||||
module axis_ctrl_crossbar_nxn_tb();
|
||||
crossbar_tb #(
|
||||
.TEST_NAME ("axis_ctrl_crossbar_nxn_tb"),
|
||||
.ROUTER_IMPL ("axis_ctrl_2d_torus" ), // Router implementation
|
||||
.ROUTER_PORTS (20 ), // Number of ports
|
||||
.ROUTER_DWIDTH (64 ), // Router datapath width
|
||||
.MTU_LOG2 (5 ), // log2 of max packet size for router
|
||||
.NUM_MASTERS (4 ), // Number of data generators in test
|
||||
.TEST_MAX_PACKETS (100 ), // How many packets to stream per test case?
|
||||
.TEST_LPP (20 ), // Lines per packet
|
||||
.TEST_MIN_INJ_RATE (10 ), // Minimum injection rate to test
|
||||
.TEST_MAX_INJ_RATE (40 ), // Maximum injection rate to test
|
||||
.TEST_INJ_RATE_INCR (10 ), // Injection rate increment
|
||||
.TEST_GEN_LL_FILES (0 ) // Generate files to produce load-latency graphs?
|
||||
) impl (
|
||||
/* no IO */
|
||||
);
|
||||
endmodule
|
||||
@@ -0,0 +1,51 @@
|
||||
#
|
||||
# Copyright 2015 Ettus Research LLC
|
||||
#
|
||||
|
||||
#-------------------------------------------------
|
||||
# Top-of-Makefile
|
||||
#-------------------------------------------------
|
||||
# Define BASE_DIR to point to the "top" dir
|
||||
BASE_DIR = $(abspath ../../../../../top)
|
||||
# Include viv_sim_preamble after defining BASE_DIR
|
||||
include $(BASE_DIR)/../tools/make/viv_sim_preamble.mak
|
||||
|
||||
#-------------------------------------------------
|
||||
# Design Specific
|
||||
#-------------------------------------------------
|
||||
# Define part using PART_ID (<device>/<package>/<speedgrade>)
|
||||
ARCH = kintex7
|
||||
PART_ID = xc7k410t/ffg900/-2
|
||||
|
||||
# Include makefiles and sources for the DUT and its dependencies
|
||||
include $(BASE_DIR)/../lib/control/Makefile.srcs
|
||||
include $(BASE_DIR)/../lib/fifo/Makefile.srcs
|
||||
include $(BASE_DIR)/../lib/rfnoc/crossbar/Makefile.srcs
|
||||
include $(BASE_DIR)/../lib/rfnoc/core/Makefile.srcs
|
||||
|
||||
DESIGN_SRCS = $(abspath \
|
||||
$(FIFO_SRCS) \
|
||||
$(CONTROL_LIB_SRCS) \
|
||||
$(RFNOC_XBAR_SRCS) \
|
||||
$(RFNOC_CORE_SRCS) \
|
||||
)
|
||||
|
||||
#-------------------------------------------------
|
||||
# Testbench Specific
|
||||
#-------------------------------------------------
|
||||
# Define only one toplevel module
|
||||
SIM_TOP = chdr_crossbar_nxn_tb
|
||||
|
||||
SIM_SRCS = \
|
||||
$(abspath chdr_crossbar_nxn_tb.sv) \
|
||||
$(abspath ../crossbar_tb.sv) \
|
||||
$(abspath ../chdr_traffic_source_sim.sv) \
|
||||
$(abspath ../chdr_traffic_sink_sim.sv)
|
||||
|
||||
#-------------------------------------------------
|
||||
# Bottom-of-Makefile
|
||||
#-------------------------------------------------
|
||||
# Include all simulator specific makefiles here
|
||||
# Each should define a unique target to simulate
|
||||
# e.g. xsim, vsim, etc and a common "clean" target
|
||||
include $(BASE_DIR)/../tools/make/viv_simulator.mak
|
||||
@@ -0,0 +1,26 @@
|
||||
//
|
||||
// Copyright 2018 Ettus Research, A National Instruments Company
|
||||
//
|
||||
// SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
|
||||
|
||||
`timescale 1ns/1ps
|
||||
|
||||
module chdr_crossbar_nxn_tb();
|
||||
crossbar_tb #(
|
||||
.TEST_NAME ("chdr_crossbar_nxn_tb"),
|
||||
.ROUTER_IMPL ("chdr_crossbar_nxn" ), // Router implementation
|
||||
.ROUTER_PORTS (10 ), // Number of ports
|
||||
.ROUTER_DWIDTH (64 ), // Router datapath width
|
||||
.MTU_LOG2 (7 ), // log2 of max packet size for router
|
||||
.NUM_MASTERS (10 ), // Number of data generators in test
|
||||
.TEST_MAX_PACKETS (100 ), // How many packets to stream per test case?
|
||||
.TEST_LPP (100 ), // Lines per packet
|
||||
.TEST_MIN_INJ_RATE (60 ), // Minimum injection rate to test
|
||||
.TEST_MAX_INJ_RATE (100 ), // Maximum injection rate to test
|
||||
.TEST_INJ_RATE_INCR (10 ), // Injection rate increment
|
||||
.TEST_GEN_LL_FILES (0 ) // Generate files to produce load-latency graphs?
|
||||
) impl (
|
||||
/* no IO */
|
||||
);
|
||||
endmodule
|
||||
@@ -0,0 +1,150 @@
|
||||
//
|
||||
// Copyright 2018 Ettus Research, A National Instruments Company
|
||||
//
|
||||
// SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
//
|
||||
// Module: chdr_traffic_sink_sim
|
||||
// Description:
|
||||
// A sink for CHDR traffic. Simulation only.
|
||||
// Accepts packets and computes the following metrics:
|
||||
// - Data integrity errors
|
||||
// - Packet latency
|
||||
// - Throughput counts
|
||||
// All metrics can optionally be written to a file to
|
||||
// generate load-latency graphs.
|
||||
|
||||
`timescale 1ns/1ps
|
||||
|
||||
`include "sim_cvita_lib.svh"
|
||||
|
||||
module chdr_traffic_sink_sim #(
|
||||
parameter WIDTH = 64,
|
||||
parameter MTU = 5,
|
||||
parameter [15:0] NODE_ID = 'd0,
|
||||
parameter [15:0] NUM_NODES = 'd16,
|
||||
parameter FILE_PATH = ".",
|
||||
parameter FLUSH_N = 4
|
||||
) (
|
||||
// Clocks and resets
|
||||
input clk,
|
||||
input rst,
|
||||
// Settings
|
||||
input [63:0] current_time,
|
||||
input start_stb,
|
||||
input [7:0] injection_rate,
|
||||
input [15:0] lines_per_pkt,
|
||||
input [7:0] traffic_patt,
|
||||
// CHDR master interface
|
||||
input [WIDTH-1:0] s_axis_tdata,
|
||||
input s_axis_tlast,
|
||||
input s_axis_tvalid,
|
||||
output s_axis_tready,
|
||||
// Metrics
|
||||
output session_active,
|
||||
output [31:0] xfer_count,
|
||||
output [31:0] pkt_count,
|
||||
output [31:0] data_err_count,
|
||||
output [31:0] route_err_count
|
||||
);
|
||||
|
||||
// Constants
|
||||
localparam integer ERR_BIT_PKT_SIZE_MISMATCH = 1;
|
||||
localparam integer ERR_BIT_PKT_DATA_MISMATCH = 2;
|
||||
localparam integer ERR_BIT_PKT_DEST_MISMATCH = 4;
|
||||
localparam integer ERR_BIT_PKT_SEQUENCE_ERR = 8;
|
||||
|
||||
cvita_slave #(.DWIDTH(WIDTH)) s_chdr (.clk(clk));
|
||||
cvita_pkt_t pkt;
|
||||
|
||||
assign s_chdr.axis.tdata = s_axis_tdata;
|
||||
assign s_chdr.axis.tlast = s_axis_tlast;
|
||||
assign s_chdr.axis.tvalid = s_axis_tvalid;
|
||||
assign s_axis_tready = s_chdr.axis.tready;
|
||||
|
||||
logic running = 0;
|
||||
integer num_data_errs = 0;
|
||||
integer num_route_errs = 0;
|
||||
logic [31:0] num_pkts_xferd = 0;
|
||||
logic [31:0] num_samps_xferd = 0;
|
||||
|
||||
assign data_err_count = num_data_errs;
|
||||
assign route_err_count = num_route_errs;
|
||||
assign xfer_count = num_samps_xferd;
|
||||
assign pkt_count = num_pkts_xferd;
|
||||
assign session_active = running;
|
||||
|
||||
integer session = 0;
|
||||
string filename;
|
||||
integer handle = 0;
|
||||
integer err = 0;
|
||||
integer bus_idle_cnt = 0;
|
||||
logic [WIDTH-1:0] i;
|
||||
|
||||
// Egress buff in source is MTU + 4
|
||||
localparam integer IDLE_TIMEOUT = (1 << (MTU + 4 + FLUSH_N));
|
||||
|
||||
initial begin: consume_blk
|
||||
// Consume infinitely
|
||||
s_chdr.reset();
|
||||
while (1) begin
|
||||
// A session begins on the posedge of start_stb
|
||||
while (~start_stb) @(posedge clk);
|
||||
session = session + 1;
|
||||
$sformat(filename, "%s/pkts_node%05d_inj%03d_lpp%05d_traffic%c_sess%04d.csv",
|
||||
FILE_PATH, NODE_ID, injection_rate, lines_per_pkt, traffic_patt, session);
|
||||
if (FILE_PATH != "") begin
|
||||
handle = $fopen(filename, "w");
|
||||
if (handle == 0) begin
|
||||
$error("Could not open file: %s", filename);
|
||||
$finish();
|
||||
end
|
||||
end
|
||||
if (handle != 0) $fdisplay(handle, "Src,Dst,Seqno,Error,Latency");
|
||||
s_chdr.reset();
|
||||
num_data_errs = 0;
|
||||
num_route_errs = 0;
|
||||
num_pkts_xferd = 0;
|
||||
num_samps_xferd = 0;
|
||||
bus_idle_cnt = 0;
|
||||
running = 1;
|
||||
while (1) begin
|
||||
// Pull packet from bus
|
||||
err = 0;
|
||||
if (~s_chdr.axis.tvalid[0]) begin
|
||||
@(posedge clk);
|
||||
bus_idle_cnt = bus_idle_cnt + 1;
|
||||
if (bus_idle_cnt <= IDLE_TIMEOUT)
|
||||
continue;
|
||||
else
|
||||
break;
|
||||
end
|
||||
s_chdr.pull_pkt(pkt, 0);
|
||||
bus_idle_cnt = 0;
|
||||
num_pkts_xferd = num_pkts_xferd + 1;
|
||||
num_samps_xferd = num_samps_xferd + lines_per_pkt;
|
||||
// Validate packet
|
||||
if (pkt.hdr.dst_sid != NODE_ID) begin
|
||||
err = err + ERR_BIT_PKT_DEST_MISMATCH;
|
||||
num_route_errs = num_route_errs + 1;
|
||||
end
|
||||
if (pkt.payload.size() != lines_per_pkt-2) begin
|
||||
err = err + ERR_BIT_PKT_SIZE_MISMATCH;
|
||||
num_data_errs = num_data_errs + 1;
|
||||
end else begin
|
||||
for (i = 'd0; i < (lines_per_pkt-2); i=i+1) begin
|
||||
if (pkt.payload[i] != i) begin
|
||||
err = err + ERR_BIT_PKT_DATA_MISMATCH;
|
||||
num_data_errs = num_data_errs + 1;
|
||||
break;
|
||||
end
|
||||
end
|
||||
end
|
||||
if (handle != 0) $fdisplay(handle, "%00d,%00d,%00d,%00d,%00d",
|
||||
pkt.hdr.src_sid, pkt.hdr.dst_sid, pkt.hdr.seqnum, err, (current_time - pkt.hdr.timestamp));
|
||||
end
|
||||
running = 0;
|
||||
if (handle != 0) $fclose(handle);
|
||||
end
|
||||
end
|
||||
|
||||
endmodule
|
||||
@@ -0,0 +1,202 @@
|
||||
//
|
||||
// Copyright 2018 Ettus Research, A National Instruments Company
|
||||
//
|
||||
// SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
//
|
||||
// Module: chdr_traffic_source_sim
|
||||
// Description:
|
||||
// A traffic generator for CHDR traffic. Simulation only.
|
||||
// Supports multiple traffic pattern and injection rates.
|
||||
//
|
||||
|
||||
`timescale 1ns/1ps
|
||||
|
||||
`include "sim_cvita_lib.svh"
|
||||
|
||||
module chdr_traffic_source_sim #(
|
||||
parameter WIDTH = 64, // Width of the AXI-Stream data bus
|
||||
parameter MTU = 5, // log2 of the max number of lines in a packet
|
||||
parameter [15:0] NODE_ID = 'd0, // Node ID for this generator
|
||||
parameter [15:0] NUM_NODES = 'd16 // Total number of generators in the application
|
||||
) (
|
||||
// Clocks and resets
|
||||
input clk, // AXI-Stream clock
|
||||
input rst, // AXI-Stream reset
|
||||
// Settings
|
||||
input [63:0] current_time, // The current value of the global timebase (synch to clk)
|
||||
input start_stb, // A strobe that indicates the start of a generation session
|
||||
input [7:0] injection_rate, // The inject rate (in percent) to simulate
|
||||
input [15:0] lines_per_pkt, // Number of lines per packet to generate
|
||||
input [7:0] traffic_patt, // The traffic pattern (see localparams below for values)
|
||||
input [31:0] num_pkts_to_send, // Number of packets to send
|
||||
// CHDR master interface
|
||||
output [WIDTH-1:0] m_axis_tdata, // AXI-Stream master tdata
|
||||
output m_axis_tlast, // AXI-Stream master tlast
|
||||
output m_axis_tvalid, // AXI-Stream master tvalid
|
||||
input m_axis_tready, // AXI-Stream master tready
|
||||
// Metrics
|
||||
output session_active, // Signal indicating if generation session is active
|
||||
output [63:0] session_duration, // Session duration (only valid after session ends)
|
||||
output [31:0] xfer_count, // Number of lines transferred (only valid after session ends)
|
||||
output [31:0] pkt_count // Number of packets transferred (only valid after session ends)
|
||||
);
|
||||
// **** Supported Traffic Patters ****
|
||||
localparam [7:0] TRAFFIC_PATT_LOOPBACK = 8'd76; //L
|
||||
localparam [7:0] TRAFFIC_PATT_NEIGHBOR = 8'd78; //N
|
||||
localparam [7:0] TRAFFIC_PATT_BIT_COMPLEMENT = 8'd67; //C
|
||||
localparam [7:0] TRAFFIC_PATT_SEQUENTIAL = 8'd83; //S
|
||||
localparam [7:0] TRAFFIC_PATT_UNIFORM = 8'd85; //U
|
||||
localparam [7:0] TRAFFIC_PATT_UNIFORM_OTHERS = 8'd79; //O
|
||||
localparam [7:0] TRAFFIC_PATT_RANDOM_PERM = 8'd82; //R
|
||||
|
||||
cvita_master #(.DWIDTH(WIDTH)) m_chdr (.clk(clk));
|
||||
axis_t #(.DWIDTH(WIDTH)) post_fifo (.clk(clk));
|
||||
axis_t #(.DWIDTH(WIDTH)) pre_gate (.clk(clk));
|
||||
cvita_hdr_t header;
|
||||
reg throttle = 1'b1;
|
||||
|
||||
logic running = 0;
|
||||
logic [31:0] curr_pkt_num = 'd0;
|
||||
logic [31:0] num_samps_xferd = 'd0;
|
||||
logic [63:0] start_time = 0;
|
||||
logic [63:0] stop_time = 0;
|
||||
logic [15:0] last_gen_sid = (NODE_ID - 16'd1);
|
||||
|
||||
assign xfer_count = num_samps_xferd;
|
||||
assign pkt_count = curr_pkt_num;
|
||||
assign session_duration = (stop_time - start_time);
|
||||
assign session_active = running;
|
||||
|
||||
// Utility function to assign SIDs based on traffic pattern
|
||||
function [15:0] gen_dst_sid;
|
||||
input [7:0] traffic_patt;
|
||||
input [15:0] last_sid;
|
||||
|
||||
if (traffic_patt == TRAFFIC_PATT_UNIFORM) begin
|
||||
gen_dst_sid = $urandom_range('d0, NUM_NODES-'d1);
|
||||
end else if (traffic_patt == TRAFFIC_PATT_UNIFORM_OTHERS) begin
|
||||
logic [31:0] rnum = $urandom_range('d0, NUM_NODES-'d2);
|
||||
if (rnum < NODE_ID)
|
||||
gen_dst_sid = rnum[15:0];
|
||||
else
|
||||
gen_dst_sid = rnum[15:0] + 16'd1;
|
||||
end else if (traffic_patt == TRAFFIC_PATT_SEQUENTIAL) begin
|
||||
gen_dst_sid = (last_sid + 16'd1) % NUM_NODES;
|
||||
end else if (traffic_patt == TRAFFIC_PATT_NEIGHBOR) begin
|
||||
gen_dst_sid = (NODE_ID + 16'd1) % NUM_NODES;
|
||||
end else if (traffic_patt == TRAFFIC_PATT_LOOPBACK) begin
|
||||
gen_dst_sid = NODE_ID;
|
||||
end else if (traffic_patt == TRAFFIC_PATT_BIT_COMPLEMENT) begin
|
||||
gen_dst_sid = (NUM_NODES - NODE_ID - 1) % NUM_NODES;
|
||||
end else if (traffic_patt == TRAFFIC_PATT_RANDOM_PERM) begin
|
||||
//TODO: Implement me
|
||||
gen_dst_sid = 0;
|
||||
end else begin
|
||||
gen_dst_sid = 'd0;
|
||||
end
|
||||
endfunction
|
||||
|
||||
// Generation loop. Push to m_chdr infinitely fast
|
||||
initial begin: gen_blk
|
||||
// Generate infinitely
|
||||
$srandom(NODE_ID + NUM_NODES);
|
||||
m_chdr.reset();
|
||||
while (1) begin
|
||||
// A generation session begins on the posedge of start_stb
|
||||
while (~start_stb) @(posedge clk);
|
||||
curr_pkt_num = 'd0;
|
||||
m_chdr.reset();
|
||||
num_samps_xferd = 'd0;
|
||||
start_time = current_time;
|
||||
running = 1;
|
||||
while (curr_pkt_num < num_pkts_to_send) begin
|
||||
header = '{
|
||||
pkt_type:DATA, has_time:1, eob:0,
|
||||
seqnum:curr_pkt_num[11:0], length:(lines_per_pkt*8),
|
||||
src_sid:NODE_ID, dst_sid:gen_dst_sid(traffic_patt, last_gen_sid),
|
||||
timestamp:0 //TS attached later
|
||||
};
|
||||
last_gen_sid = header.dst_sid;
|
||||
curr_pkt_num = curr_pkt_num + 'd1;
|
||||
m_chdr.push_ramp_pkt(lines_per_pkt-2, 'h0, 'h1, header);
|
||||
num_samps_xferd = num_samps_xferd + lines_per_pkt;
|
||||
end
|
||||
running = 0;
|
||||
stop_time = current_time;
|
||||
end
|
||||
end
|
||||
|
||||
// Capture packets in a really short FIFO (for backpressure)
|
||||
axi_fifo #(
|
||||
.WIDTH(WIDTH+1), .SIZE(MTU + 1)
|
||||
) fifo_i (
|
||||
.clk (clk),
|
||||
.reset (rst),
|
||||
.clear (1'b0),
|
||||
.i_tdata ({m_chdr.axis.tlast, m_chdr.axis.tdata}),
|
||||
.i_tvalid (m_chdr.axis.tvalid),
|
||||
.i_tready (m_chdr.axis.tready),
|
||||
.o_tdata ({post_fifo.tlast, post_fifo.tdata}),
|
||||
.o_tvalid (post_fifo.tvalid),
|
||||
.o_tready (post_fifo.tready),
|
||||
.space (),
|
||||
.occupied ()
|
||||
);
|
||||
|
||||
// Attach timestamp after the packet leaves the FIFO after
|
||||
// throttling.
|
||||
|
||||
localparam [1:0] ST_HDR = 2'd0;
|
||||
localparam [1:0] ST_TS = 2'd1;
|
||||
localparam [1:0] ST_BODY = 2'd2;
|
||||
|
||||
reg [1:0] pkt_state = ST_HDR;
|
||||
always_ff @(posedge clk) begin
|
||||
if (rst) begin
|
||||
pkt_state <= ST_HDR;
|
||||
end else if (pre_gate.tvalid & pre_gate.tready) begin
|
||||
case (pkt_state)
|
||||
ST_HDR:
|
||||
if (~pre_gate.tlast)
|
||||
pkt_state <= pre_gate.tdata[61] ? ST_TS : ST_BODY;
|
||||
ST_TS:
|
||||
pkt_state <= pre_gate.tlast ? ST_HDR : ST_BODY;
|
||||
ST_BODY:
|
||||
pkt_state <= pre_gate.tlast ? ST_HDR : ST_BODY;
|
||||
default:
|
||||
pkt_state <= ST_HDR;
|
||||
endcase
|
||||
end
|
||||
end
|
||||
|
||||
// Enforce injection rate by pulling from FIFO with a certain time probability
|
||||
always_ff @(posedge clk) begin
|
||||
throttle <= ($urandom_range(32'd99, 32'd0) > {24'h0, injection_rate});
|
||||
end
|
||||
|
||||
// Insert timestamp + throttle logic
|
||||
assign pre_gate.tdata = (pkt_state == ST_TS) ? current_time : post_fifo.tdata;
|
||||
assign pre_gate.tlast = post_fifo.tlast;
|
||||
assign pre_gate.tvalid = post_fifo.tvalid & ~throttle;
|
||||
assign post_fifo.tready = pre_gate.tready & ~throttle;
|
||||
|
||||
// Gate the packet to smooth out throttle-related noise.
|
||||
// This also serves as a buffer for the packet in case things are backed up
|
||||
axi_packet_gate #(
|
||||
.WIDTH(WIDTH), .SIZE(MTU + 4), .USE_AS_BUFF(1)
|
||||
) pkt_gate_i (
|
||||
.clk (clk),
|
||||
.reset (rst),
|
||||
.clear (1'b0),
|
||||
.i_tdata (pre_gate.tdata),
|
||||
.i_tlast (pre_gate.tlast),
|
||||
.i_terror (1'b0),
|
||||
.i_tvalid (pre_gate.tvalid),
|
||||
.i_tready (pre_gate.tready),
|
||||
.o_tdata (m_axis_tdata),
|
||||
.o_tlast (m_axis_tlast),
|
||||
.o_tvalid (m_axis_tvalid),
|
||||
.o_tready (m_axis_tready)
|
||||
);
|
||||
|
||||
endmodule
|
||||
@@ -0,0 +1,428 @@
|
||||
//
|
||||
// Copyright 2018 Ettus Research, A National Instruments Company
|
||||
//
|
||||
// SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
|
||||
|
||||
`timescale 1ns/1ps
|
||||
`define NS_PER_TICK 1
|
||||
`define NUM_TEST_CASES 7
|
||||
|
||||
`include "sim_clks_rsts.vh"
|
||||
`include "sim_exec_report.vh"
|
||||
`include "sim_set_rb_lib.svh"
|
||||
`include "sim_axis_lib.svh"
|
||||
|
||||
`define SIM_TIMEOUT_US 1000000 // Default: 1s
|
||||
|
||||
module crossbar_tb #(
|
||||
parameter TEST_NAME = "crossbar_tb",
|
||||
// Router parameters
|
||||
parameter ROUTER_IMPL = "axi_crossbar", // Router implementation
|
||||
parameter ROUTER_PORTS = 10, // # Router ports
|
||||
parameter ROUTER_DWIDTH = 64, // Router datapath width
|
||||
parameter MTU_LOG2 = 7, // log2 of max packet size for router
|
||||
parameter NUM_MASTERS = ROUTER_PORTS, // Number of data generators in test
|
||||
// Test parameters
|
||||
parameter TEST_MAX_PACKETS = 50, // How many packets to stream per test case?
|
||||
parameter TEST_LPP = 50, // Lines per packet
|
||||
parameter TEST_MIN_INJ_RATE = 60, // Minimum injection rate to test
|
||||
parameter TEST_MAX_INJ_RATE = 100, // Maximum injection rate to test
|
||||
parameter TEST_INJ_RATE_INCR = 10, // Injection rate increment
|
||||
parameter TEST_GEN_LL_FILES = 0 // Generate files to produce load-latency graphs?
|
||||
|
||||
)(
|
||||
/* no IO */
|
||||
);
|
||||
`TEST_BENCH_INIT(TEST_NAME,`NUM_TEST_CASES,`NS_PER_TICK)
|
||||
|
||||
//----------------------------------------------------
|
||||
// General test setup
|
||||
//----------------------------------------------------
|
||||
|
||||
// Clocks and reset
|
||||
`DEFINE_CLK(clk, 5.000, 50)
|
||||
`DEFINE_RESET(rst, 0, 10)
|
||||
|
||||
// Timekeeper (cycle counter)
|
||||
logic [63:0] timestamp;
|
||||
initial begin : timekeeper_blk
|
||||
while (rst) @(posedge clk);
|
||||
timestamp = 'd0;
|
||||
while (~rst) begin
|
||||
@(posedge clk);
|
||||
timestamp = timestamp + 'd1;
|
||||
end
|
||||
end
|
||||
|
||||
//----------------------------------------------------
|
||||
// Instantiate traffic generators, checkers, buses
|
||||
//----------------------------------------------------
|
||||
localparam FILE_PATH = {`WORKING_DIR, "/data/", ROUTER_IMPL};
|
||||
|
||||
// Data buses
|
||||
axis_t #(.DWIDTH(ROUTER_DWIDTH), .NUM_STREAMS(ROUTER_PORTS)) src2rtr_axis (.clk(clk));
|
||||
axis_t #(.DWIDTH(ROUTER_DWIDTH), .NUM_STREAMS(ROUTER_PORTS)) rtr2snk_axis (.clk(clk));
|
||||
|
||||
// Control buses
|
||||
settings_bus_master #(.SR_AWIDTH(16), .SR_DWIDTH(32)) rtr_sb (.clk(clk));
|
||||
wire rtr_sb_ack;
|
||||
|
||||
// Test vector source and sink instantiation
|
||||
logic [7:0] set_injection_rate;
|
||||
logic [15:0] set_lines_per_pkt;
|
||||
logic [7:0] set_traffic_patt;
|
||||
logic [31:0] set_num_pkts_to_send;
|
||||
logic snk_start_stb = 0;
|
||||
logic src_start_stb = 0;
|
||||
|
||||
wire [63:0] session_duration [0:ROUTER_PORTS-1];
|
||||
wire [ROUTER_PORTS-1:0] src_active;
|
||||
wire [31:0] src_xfer_count [0:ROUTER_PORTS-1];
|
||||
wire [31:0] src_pkt_count [0:ROUTER_PORTS-1];
|
||||
wire [ROUTER_PORTS-1:0] snk_active;
|
||||
wire [31:0] snk_xfer_count [0:ROUTER_PORTS-1];
|
||||
wire [31:0] snk_pkt_count [0:ROUTER_PORTS-1];
|
||||
wire [31:0] snk_data_err_count [0:ROUTER_PORTS-1];
|
||||
wire [31:0] snk_route_err_count[0:ROUTER_PORTS-1];
|
||||
|
||||
wire deadlock_detected;
|
||||
reg deadlock_detected_del = 1'b0;
|
||||
always @(posedge clk) deadlock_detected_del <= deadlock_detected;
|
||||
wire deadlock_re = (deadlock_detected & ~deadlock_detected_del);
|
||||
wire deadlock_fe = (~deadlock_detected & deadlock_detected_del);
|
||||
|
||||
genvar i;
|
||||
generate for (i = 0; i < ROUTER_PORTS; i=i+1) begin: src_snk_blk
|
||||
chdr_traffic_source_sim #(
|
||||
.WIDTH (ROUTER_DWIDTH),
|
||||
.MTU (MTU_LOG2),
|
||||
.NODE_ID (i),
|
||||
.NUM_NODES (ROUTER_PORTS)
|
||||
) traffic_src (
|
||||
.clk (clk),
|
||||
.rst (rst),
|
||||
.current_time (timestamp),
|
||||
.start_stb (src_start_stb & (i < NUM_MASTERS)),
|
||||
.injection_rate (set_injection_rate),
|
||||
.lines_per_pkt (set_lines_per_pkt),
|
||||
.traffic_patt (set_traffic_patt),
|
||||
.num_pkts_to_send (set_num_pkts_to_send),
|
||||
.m_axis_tdata (src2rtr_axis.tdata[((i+1)*ROUTER_DWIDTH)-1:i*ROUTER_DWIDTH]),
|
||||
.m_axis_tlast (src2rtr_axis.tlast[i]),
|
||||
.m_axis_tvalid (src2rtr_axis.tvalid[i]),
|
||||
.m_axis_tready (src2rtr_axis.tready[i]),
|
||||
.session_active (src_active[i]),
|
||||
.session_duration (session_duration[i]),
|
||||
.xfer_count (src_xfer_count[i]),
|
||||
.pkt_count (src_pkt_count[i])
|
||||
);
|
||||
|
||||
chdr_traffic_sink_sim #(
|
||||
.WIDTH (ROUTER_DWIDTH),
|
||||
.MTU (MTU_LOG2),
|
||||
.NODE_ID (i),
|
||||
.NUM_NODES (ROUTER_PORTS),
|
||||
.FILE_PATH (TEST_GEN_LL_FILES==1 ? FILE_PATH : "")
|
||||
) traffic_sink (
|
||||
.clk (clk),
|
||||
.rst (rst),
|
||||
.current_time (timestamp),
|
||||
.start_stb (snk_start_stb),
|
||||
.injection_rate (set_injection_rate),
|
||||
.lines_per_pkt (set_lines_per_pkt),
|
||||
.traffic_patt (set_traffic_patt),
|
||||
.s_axis_tdata (rtr2snk_axis.tdata[((i+1)*ROUTER_DWIDTH)-1:i*ROUTER_DWIDTH]),
|
||||
.s_axis_tlast (rtr2snk_axis.tlast[i]),
|
||||
.s_axis_tvalid (rtr2snk_axis.tvalid[i]),
|
||||
.s_axis_tready (rtr2snk_axis.tready[i]),
|
||||
.session_active (snk_active[i]),
|
||||
.xfer_count (snk_xfer_count[i]),
|
||||
.pkt_count (snk_pkt_count[i]),
|
||||
.data_err_count (snk_data_err_count[i]),
|
||||
.route_err_count (snk_route_err_count[i])
|
||||
);
|
||||
end endgenerate
|
||||
|
||||
//----------------------------------------------------
|
||||
// Instantiate DUT
|
||||
//----------------------------------------------------
|
||||
generate if (ROUTER_IMPL == "FIFO") begin
|
||||
for (i = 0; i < ROUTER_PORTS; i=i+1) begin
|
||||
axi_fifo #(
|
||||
.WIDTH(ROUTER_DWIDTH+1), .SIZE(0)
|
||||
) fifo_i (
|
||||
.clk (clk),
|
||||
.reset (rst),
|
||||
.clear (1'b0),
|
||||
.i_tdata ({src2rtr_axis.tlast[i], src2rtr_axis.tdata[((i+1)*ROUTER_DWIDTH)-1:i*ROUTER_DWIDTH]}),
|
||||
.i_tvalid (src2rtr_axis.tvalid[i]),
|
||||
.i_tready (src2rtr_axis.tready[i]),
|
||||
.o_tdata ({rtr2snk_axis.tlast[i], rtr2snk_axis.tdata[((i+1)*ROUTER_DWIDTH)-1:i*ROUTER_DWIDTH]}),
|
||||
.o_tvalid (rtr2snk_axis.tvalid[i]),
|
||||
.o_tready (rtr2snk_axis.tready[i]),
|
||||
.space (),
|
||||
.occupied ()
|
||||
);
|
||||
end
|
||||
end else if (ROUTER_IMPL == "axi_crossbar") begin
|
||||
axi_crossbar #(
|
||||
.BASE (0),
|
||||
.FIFO_WIDTH (ROUTER_DWIDTH),
|
||||
.DST_WIDTH (16),
|
||||
.NUM_INPUTS (ROUTER_PORTS),
|
||||
.NUM_OUTPUTS (ROUTER_PORTS)
|
||||
) router_dut_i (
|
||||
// General
|
||||
.clk (clk),
|
||||
.reset (rst),
|
||||
.clear (1'b0),
|
||||
.local_addr (8'd0),
|
||||
// Inputs
|
||||
.i_tdata (src2rtr_axis.tdata),
|
||||
.i_tlast (src2rtr_axis.tlast),
|
||||
.i_tvalid (src2rtr_axis.tvalid),
|
||||
.i_tready (src2rtr_axis.tready),
|
||||
.pkt_present (src2rtr_axis.tvalid),
|
||||
// Output
|
||||
.o_tdata (rtr2snk_axis.tdata),
|
||||
.o_tlast (rtr2snk_axis.tlast),
|
||||
.o_tvalid (rtr2snk_axis.tvalid),
|
||||
.o_tready (rtr2snk_axis.tready),
|
||||
// Setting Bus
|
||||
.set_stb (rtr_sb.settings_bus.set_stb),
|
||||
.set_addr (rtr_sb.settings_bus.set_addr),
|
||||
.set_data (rtr_sb.settings_bus.set_data),
|
||||
// Readback bus
|
||||
.rb_rd_stb (1'b0),
|
||||
.rb_addr ({(2*$clog2(ROUTER_PORTS)){1'b0}}),
|
||||
.rb_data ()
|
||||
);
|
||||
end else if (ROUTER_IMPL == "chdr_crossbar_nxn") begin
|
||||
chdr_crossbar_nxn #(
|
||||
.CHDR_W (ROUTER_DWIDTH),
|
||||
.NPORTS (ROUTER_PORTS),
|
||||
.DEFAULT_PORT (0),
|
||||
.MTU (MTU_LOG2),
|
||||
.ROUTE_TBL_SIZE (6),
|
||||
.MUX_ALLOC ("ROUND-ROBIN"),
|
||||
.OPTIMIZE ("AREA"),
|
||||
.NPORTS_MGMT (0),
|
||||
.EXT_RTCFG_PORT (1)
|
||||
) router_dut_i (
|
||||
// General
|
||||
.clk (clk),
|
||||
.reset (rst),
|
||||
// Inputs
|
||||
.s_axis_tdata (src2rtr_axis.tdata),
|
||||
.s_axis_tlast (src2rtr_axis.tlast),
|
||||
.s_axis_tvalid (src2rtr_axis.tvalid),
|
||||
.s_axis_tready (src2rtr_axis.tready),
|
||||
// Output
|
||||
.m_axis_tdata (rtr2snk_axis.tdata),
|
||||
.m_axis_tlast (rtr2snk_axis.tlast),
|
||||
.m_axis_tvalid (rtr2snk_axis.tvalid),
|
||||
.m_axis_tready (rtr2snk_axis.tready),
|
||||
// External router config
|
||||
.ext_rtcfg_stb (rtr_sb.settings_bus.set_stb),
|
||||
.ext_rtcfg_addr (rtr_sb.settings_bus.set_addr),
|
||||
.ext_rtcfg_data (rtr_sb.settings_bus.set_data),
|
||||
.ext_rtcfg_ack (rtr_sb_ack)
|
||||
);
|
||||
end else begin
|
||||
axis_ctrl_crossbar_nxn #(
|
||||
.WIDTH (ROUTER_DWIDTH),
|
||||
.NPORTS (ROUTER_PORTS),
|
||||
.TOPOLOGY (ROUTER_IMPL == "axis_ctrl_2d_torus" ? "TORUS" : "MESH"),
|
||||
.INGRESS_BUFF_SIZE(MTU_LOG2),
|
||||
.ROUTER_BUFF_SIZE (MTU_LOG2),
|
||||
.ROUTING_ALLOC ("WORMHOLE"),
|
||||
.SWITCH_ALLOC ("PRIO")
|
||||
) router_dut_i (
|
||||
// General
|
||||
.clk (clk),
|
||||
.reset (rst),
|
||||
// Inputs
|
||||
.s_axis_tdata (src2rtr_axis.tdata),
|
||||
.s_axis_tlast (src2rtr_axis.tlast),
|
||||
.s_axis_tvalid (src2rtr_axis.tvalid),
|
||||
.s_axis_tready (src2rtr_axis.tready),
|
||||
// Output
|
||||
.m_axis_tdata (rtr2snk_axis.tdata),
|
||||
.m_axis_tlast (rtr2snk_axis.tlast),
|
||||
.m_axis_tvalid (rtr2snk_axis.tvalid),
|
||||
.m_axis_tready (rtr2snk_axis.tready),
|
||||
// Deadlock detection
|
||||
.deadlock_detected(deadlock_detected)
|
||||
);
|
||||
end endgenerate
|
||||
|
||||
//----------------------------------------------------
|
||||
// Test routine. Runs tests and writes metrics to file
|
||||
//----------------------------------------------------
|
||||
|
||||
// Constants
|
||||
localparam [7:0] TRAFFIC_PATT_LOOPBACK = 8'd76; //L
|
||||
localparam [7:0] TRAFFIC_PATT_NEIGHBOR = 8'd78; //N
|
||||
localparam [7:0] TRAFFIC_PATT_BIT_COMPLEMENT = 8'd67; //C
|
||||
localparam [7:0] TRAFFIC_PATT_SEQUENTIAL = 8'd83; //S
|
||||
localparam [7:0] TRAFFIC_PATT_UNIFORM = 8'd85; //U
|
||||
localparam [7:0] TRAFFIC_PATT_UNIFORM_OTHERS = 8'd79; //O
|
||||
localparam [7:0] TRAFFIC_PATT_RANDOM_PERM = 8'd82; //R
|
||||
|
||||
string filename;
|
||||
integer node;
|
||||
integer session = 0;
|
||||
integer handle = 0;
|
||||
logic [63:0] start_time;
|
||||
integer total_pkts_recvd = 0, total_pkts_sent = 0;
|
||||
|
||||
task sim_dataflow;
|
||||
input [7:0] injection_rate;
|
||||
input [7:0] traffic_patt;
|
||||
input [15:0] lines_per_pkt;
|
||||
input [31:0] num_pkts_to_send;
|
||||
begin
|
||||
session = session + 1;
|
||||
$display("--------------- New Simulation ---------------");
|
||||
$display("- Module = %s", ROUTER_IMPL);
|
||||
$display("- Nodes = %00d", ROUTER_PORTS);
|
||||
$display("- Injection Rate = %00d%%", injection_rate);
|
||||
$display("- Traffic Pattern = %c", traffic_patt);
|
||||
$display("- Packet Size = %00d words (%00d bits)", lines_per_pkt, ROUTER_DWIDTH);
|
||||
$display("- Max Packets = %00d", num_pkts_to_send);
|
||||
// Configure settings
|
||||
@(posedge clk);
|
||||
set_injection_rate = injection_rate;
|
||||
set_lines_per_pkt = lines_per_pkt;
|
||||
set_traffic_patt = traffic_patt;
|
||||
set_num_pkts_to_send = num_pkts_to_send;
|
||||
@(posedge clk);
|
||||
// Start the sink then the source
|
||||
$display("Data flow starting...");
|
||||
snk_start_stb = 1;
|
||||
src_start_stb = 1;
|
||||
@(posedge clk);
|
||||
src_start_stb = 0;
|
||||
snk_start_stb = 0;
|
||||
@(posedge clk);
|
||||
start_time = timestamp;
|
||||
// Wait for source blocks to finish generating
|
||||
$display("Waiting for packets to transmit... (may take a while)");
|
||||
while (|src_active) begin
|
||||
@(posedge clk);
|
||||
if (deadlock_re) $display("WARNING: Deadlock detected");
|
||||
if (deadlock_fe) $display("Recovered from deadlock");
|
||||
end
|
||||
// Wait for sink blocks to finish consuming
|
||||
$display("All packets transmitted. Waiting to flush...");
|
||||
while (|snk_active) @(posedge clk);
|
||||
// If router deadlocks then wait for it to recover
|
||||
if (deadlock_detected) begin
|
||||
$display("Waiting for deadlock recovery to finish...");
|
||||
while (deadlock_detected) @(posedge clk);
|
||||
end
|
||||
repeat(set_lines_per_pkt) @(posedge clk);
|
||||
// Record summary to file and print to console
|
||||
$sformat(filename, "%s/info_inj%03d_lpp%05d_traffic%c_sess%04d.csv",
|
||||
FILE_PATH, injection_rate, lines_per_pkt, traffic_patt, session);
|
||||
if (TEST_GEN_LL_FILES == 1) begin
|
||||
handle = $fopen(filename, "w");
|
||||
if (handle == 0) begin
|
||||
$error("Could not open file: %s", filename);
|
||||
$finish();
|
||||
end
|
||||
end
|
||||
if (handle != 0) $fdisplay(handle, "Impl,Node,TxPkts,RxPkts,Duration,ErrRoute,ErrData");
|
||||
total_pkts_sent = 0;
|
||||
total_pkts_recvd = 0;
|
||||
for (node = 0; node < ROUTER_PORTS; node=node+1) begin
|
||||
$display("- Node #%03d: TX = %5d pkts, RX = %5d pkts, Inj Rate = %3d%%. Errs = %5d route, %5d data",
|
||||
node,src_pkt_count[node], snk_pkt_count[node], ((src_xfer_count[node]*100)/session_duration[node]),
|
||||
snk_route_err_count[node], snk_data_err_count[node]);
|
||||
if (handle != 0) $fdisplay(handle, "%s,%00d,%00d,%00d,%00d,%00d,%00d", ROUTER_IMPL,
|
||||
node,src_pkt_count[node], snk_pkt_count[node], session_duration[node],
|
||||
snk_route_err_count[node], snk_data_err_count[node]);
|
||||
total_pkts_sent = total_pkts_sent + src_pkt_count[node];
|
||||
total_pkts_recvd = total_pkts_recvd + snk_pkt_count[node];
|
||||
`ASSERT_ERROR(snk_route_err_count[node] == 0, "Routing errors. Received packets destined to other nodes");
|
||||
`ASSERT_ERROR(snk_data_err_count[node] == 0, "Integrity errors. Received corrupted packets");
|
||||
end
|
||||
$display("Finished. Elapsed = %00d cycles, TX = %00d pkts, RX = %00d pkts",
|
||||
(timestamp - start_time), total_pkts_sent, total_pkts_recvd);
|
||||
`ASSERT_ERROR(total_pkts_recvd == total_pkts_sent, "Total # TX packets did not match the total # RX packets");
|
||||
if (handle != 0) $fclose(handle);
|
||||
$display("----------------------------------------------");
|
||||
end
|
||||
endtask
|
||||
|
||||
//----------------------------------------------------
|
||||
// Main test loop
|
||||
//----------------------------------------------------
|
||||
|
||||
logic [31:0] MAX_PACKETS = TEST_MAX_PACKETS;
|
||||
logic [15:0] LPP = TEST_LPP;
|
||||
integer MIN_INJ_RATE = TEST_MIN_INJ_RATE;
|
||||
integer MAX_INJ_RATE = TEST_MAX_INJ_RATE;
|
||||
integer INJ_RATE_INCR = TEST_INJ_RATE_INCR;
|
||||
|
||||
integer inj_rate = 0;
|
||||
initial begin : tb_main
|
||||
src_start_stb = 0;
|
||||
snk_start_stb = 0;
|
||||
rtr_sb.reset();
|
||||
while (rst) @(posedge clk);
|
||||
|
||||
repeat (10) @(posedge clk);
|
||||
|
||||
`TEST_CASE_START("Set up crossbar");
|
||||
for (node = 0; node < ROUTER_PORTS; node=node+1) begin
|
||||
if (ROUTER_IMPL == "axi_crossbar") begin
|
||||
rtr_sb.write(16'd256 + node[15:0], {16'h0, node[15:0]});
|
||||
end else if (ROUTER_IMPL == "chdr_crossbar_nxn") begin
|
||||
rtr_sb.write(node[15:0], {16'h0, node[15:0]});
|
||||
while (~rtr_sb_ack) @(posedge clk);
|
||||
end
|
||||
end
|
||||
`TEST_CASE_DONE(1)
|
||||
|
||||
`TEST_CASE_START("Simulate LOOPBACK Traffic Pattern");
|
||||
for (inj_rate = MIN_INJ_RATE; inj_rate <= MAX_INJ_RATE; inj_rate = inj_rate + INJ_RATE_INCR) begin
|
||||
sim_dataflow(inj_rate, TRAFFIC_PATT_LOOPBACK, LPP, MAX_PACKETS);
|
||||
end
|
||||
`TEST_CASE_DONE(1)
|
||||
|
||||
`TEST_CASE_START("Simulate SEQUENTIAL Traffic Pattern");
|
||||
for (inj_rate = MIN_INJ_RATE; inj_rate <= MAX_INJ_RATE; inj_rate = inj_rate + INJ_RATE_INCR) begin
|
||||
sim_dataflow(inj_rate, TRAFFIC_PATT_SEQUENTIAL, LPP, MAX_PACKETS);
|
||||
end
|
||||
`TEST_CASE_DONE(1)
|
||||
|
||||
`TEST_CASE_START("Simulate UNIFORM Traffic Pattern");
|
||||
for (inj_rate = MIN_INJ_RATE; inj_rate <= MAX_INJ_RATE; inj_rate = inj_rate + INJ_RATE_INCR) begin
|
||||
sim_dataflow(inj_rate, TRAFFIC_PATT_UNIFORM, LPP, MAX_PACKETS);
|
||||
end
|
||||
`TEST_CASE_DONE(1)
|
||||
|
||||
`TEST_CASE_START("Simulate UNIFORM_OTHERS Traffic Pattern");
|
||||
for (inj_rate = MIN_INJ_RATE; inj_rate <= MAX_INJ_RATE; inj_rate = inj_rate + INJ_RATE_INCR) begin
|
||||
sim_dataflow(inj_rate, TRAFFIC_PATT_UNIFORM_OTHERS, LPP, MAX_PACKETS);
|
||||
end
|
||||
`TEST_CASE_DONE(1)
|
||||
|
||||
`TEST_CASE_START("Simulate BIT_COMPLEMENT Traffic Pattern");
|
||||
for (inj_rate = MIN_INJ_RATE; inj_rate <= MAX_INJ_RATE; inj_rate = inj_rate + INJ_RATE_INCR) begin
|
||||
sim_dataflow(inj_rate, TRAFFIC_PATT_BIT_COMPLEMENT, LPP, MAX_PACKETS);
|
||||
end
|
||||
`TEST_CASE_DONE(1)
|
||||
|
||||
`TEST_CASE_START("Simulate NEIGHBOR Traffic Pattern");
|
||||
for (inj_rate = MIN_INJ_RATE; inj_rate <= MAX_INJ_RATE; inj_rate = inj_rate + INJ_RATE_INCR) begin
|
||||
sim_dataflow(inj_rate, TRAFFIC_PATT_NEIGHBOR, LPP, MAX_PACKETS);
|
||||
end
|
||||
`TEST_CASE_DONE(1)
|
||||
|
||||
`TEST_BENCH_DONE
|
||||
end // initial begin
|
||||
|
||||
endmodule
|
||||
+169
@@ -0,0 +1,169 @@
|
||||
#!/usr/bin/env python3
|
||||
#
|
||||
# Copyright 2018 Ettus Research, A National Instruments Company
|
||||
#
|
||||
# SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
#
|
||||
# Description
|
||||
# Parses the output files generated by crossbar_tb and outputs
|
||||
# a load-latency graph and a expected-actual throughput graph
|
||||
|
||||
import os, sys
|
||||
import argparse
|
||||
import time
|
||||
import glob
|
||||
import csv
|
||||
import re
|
||||
import numpy as np
|
||||
|
||||
import matplotlib
|
||||
#matplotlib.use('Agg')
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
def get_options():
|
||||
parser = argparse.ArgumentParser(description='Generate Load Latency Graphs')
|
||||
parser.add_argument('datadir', type=str, default='.', help='Location of packet capture files generated by testbench')
|
||||
return parser.parse_args()
|
||||
|
||||
TRAFFIC_PATTERNS = {'U':'UNIFORM', 'O':'UNIFORM_OTHERS', 'N':'NEIGHBOR', 'L':'LOOPBACK', 'S':'SEQUENTIAL', 'C':'BIT_COMPLEMENT', 'R':'RANDOM_PERM'}
|
||||
|
||||
class InfoFile():
|
||||
def __init__(self, filename):
|
||||
# Extract test info from filename
|
||||
m = re.search(r".*/info_inj([0-9]+)_lpp([0-9]+)_traffic(.)_sess([0-9]+)\.csv", filename)
|
||||
if m is None:
|
||||
raise ValueError('Incorrect filename format: %s'%(filename))
|
||||
self.inj_rate = int(m.group(1))
|
||||
self.lpp = int(m.group(2))
|
||||
self.traffic_patt = TRAFFIC_PATTERNS[m.group(3)]
|
||||
self.session = int(m.group(4))
|
||||
|
||||
self.tx_pkts = 0
|
||||
self.rx_pkts = 0
|
||||
self.duration = 0
|
||||
self.errs = 0
|
||||
self.nodes = 0
|
||||
with open(filename, 'r') as csvfile:
|
||||
reader = csv.reader(csvfile, delimiter=',')
|
||||
isheader = True
|
||||
for row in reader:
|
||||
if isheader:
|
||||
isheader = False
|
||||
if row != ['Impl', 'Node', 'TxPkts', 'RxPkts', 'Duration', 'ErrRoute', 'ErrData']:
|
||||
raise ValueError('Incorrect header: %s'%(filename))
|
||||
else:
|
||||
self.impl = row[0]
|
||||
self.tx_pkts = self.tx_pkts + int(row[2])
|
||||
self.rx_pkts = self.tx_pkts + int(row[3])
|
||||
self.duration = self.duration + int(row[4])
|
||||
self.errs = self.errs + int(row[5]) + int(row[6])
|
||||
self.nodes = self.nodes + 1
|
||||
self.real_inj_rate = (100.0 * self.tx_pkts * self.lpp) / self.duration
|
||||
|
||||
class PktFile():
|
||||
def __init__(self, filename):
|
||||
# Extract test info from filename
|
||||
m = re.search(r".*/pkts_node([0-9]+)_inj([0-9]+)_lpp([0-9]+)_traffic(.)_sess([0-9]+)\.csv", filename)
|
||||
if m is None:
|
||||
raise ValueError('Incorrect filename format: %s'%(filename))
|
||||
self.node = int(m.group(1))
|
||||
self.inj_rate = int(m.group(2))
|
||||
self.lpp = int(m.group(3))
|
||||
self.traffic_patt = TRAFFIC_PATTERNS[m.group(4)]
|
||||
self.session = int(m.group(5))
|
||||
|
||||
self.latencies = []
|
||||
with open(filename, 'r') as csvfile:
|
||||
reader = csv.reader(csvfile, delimiter=',')
|
||||
isheader = True
|
||||
for row in reader:
|
||||
if isheader:
|
||||
isheader = False
|
||||
if row != ['Src', 'Dst', 'Seqno', 'Error', 'Latency']:
|
||||
raise ValueError('Incorrect header: %s'%(filename))
|
||||
else:
|
||||
self.latencies.append(int(row[4]))
|
||||
|
||||
|
||||
########################################################################
|
||||
# main
|
||||
########################################################################
|
||||
if __name__=='__main__':
|
||||
options = get_options()
|
||||
|
||||
if (not os.path.isdir(options.datadir)):
|
||||
print('ERROR: Data director %s does not exist'%(options.datadir))
|
||||
sys.exit(1)
|
||||
|
||||
info_db = dict()
|
||||
info_files = glob.glob(os.path.join(options.datadir, 'info*.csv'))
|
||||
router_impl = ''
|
||||
lines_per_pkt = 0
|
||||
for ifile in info_files:
|
||||
print('INFO: Reading %s...'%(ifile))
|
||||
tmp = InfoFile(ifile)
|
||||
router_impl = tmp.impl # Assume that all files have the same impl
|
||||
lines_per_pkt = tmp.lpp # Assume that all files have the same LPP
|
||||
info_db[(tmp.lpp, tmp.traffic_patt, tmp.inj_rate)] = tmp
|
||||
|
||||
pkt_db = dict()
|
||||
pkts_files = glob.glob(os.path.join(options.datadir, 'pkts*.csv'))
|
||||
for pfile in pkts_files:
|
||||
print('INFO: Reading %s...'%(pfile))
|
||||
tmp = PktFile(pfile)
|
||||
config_key = (tmp.lpp, tmp.traffic_patt)
|
||||
if config_key not in pkt_db:
|
||||
pkt_db[config_key] = dict()
|
||||
if tmp.inj_rate not in pkt_db[config_key]:
|
||||
pkt_db[config_key][tmp.inj_rate] = []
|
||||
|
||||
|
||||
pkt_db[config_key][tmp.inj_rate].extend(tmp.latencies)
|
||||
|
||||
# Write load-latency plots to file
|
||||
actual_inj_rate_db = dict()
|
||||
for config in sorted(pkt_db):
|
||||
(lpp, traffic_patt) = config
|
||||
ll_file = 'load-latency_%s_traffic-%s_lpp-%d.png'%(router_impl, traffic_patt, lpp)
|
||||
print('INFO: Writing file ' + ll_file + '...')
|
||||
percentile = [0, 25, 50, 75, 90, 95, 99, 99.9, 100]
|
||||
plt.figure()
|
||||
plt.title('Load Latency Graph for %s\n(Traffic: %s, LPP: %d)'%(router_impl, traffic_patt, lpp))
|
||||
for p in percentile:
|
||||
plot_data = dict()
|
||||
for inj_rate in pkt_db[config]:
|
||||
real_inj_rate = info_db[(lpp, traffic_patt, inj_rate)].real_inj_rate
|
||||
plot_data[real_inj_rate] = np.percentile(pkt_db[config][inj_rate], p)
|
||||
latencies = []
|
||||
rates = []
|
||||
for inj_rate in sorted(plot_data):
|
||||
rates.append(inj_rate)
|
||||
latencies.append(plot_data[inj_rate])
|
||||
plt.plot(rates, latencies, label='$P_{%.1f}$'%(p))
|
||||
plt.xlabel('Load (%)')
|
||||
plt.xticks(range(0, 110, 10))
|
||||
plt.ylabel('Latency (cycles)')
|
||||
plt.grid(True)
|
||||
plt.legend()
|
||||
plt.savefig(os.path.join(options.datadir, ll_file), dpi=120)
|
||||
# Generate actual inj_rate graph
|
||||
real_inj_rates = []
|
||||
for inj_rate in sorted(pkt_db[config]):
|
||||
real_inj_rates.append(info_db[(lpp, traffic_patt, inj_rate)].real_inj_rate)
|
||||
actual_inj_rate_db[config] = (sorted(pkt_db[config]), real_inj_rates)
|
||||
|
||||
# Write offered vs actual injection rate plots to file
|
||||
injrate_file = 'injection-rate_%s_lpp-%d.png'%(router_impl, lines_per_pkt)
|
||||
print('INFO: Writing file ' + injrate_file + '...')
|
||||
plt.figure()
|
||||
plt.title('Max Injection Rate Graph for %s'%(router_impl))
|
||||
for config in actual_inj_rate_db:
|
||||
(x, y) = actual_inj_rate_db[config]
|
||||
plt.plot(x, y, label=str(config))
|
||||
plt.xlabel('Offered Injection Rate (%)')
|
||||
plt.xticks(range(0, 110, 10))
|
||||
plt.ylabel('Accepted Injection Rate (%)')
|
||||
plt.yticks(range(0, 110, 10))
|
||||
plt.grid(True)
|
||||
plt.legend()
|
||||
plt.savefig(os.path.join(options.datadir, injrate_file), dpi=120)
|
||||
+106
@@ -0,0 +1,106 @@
|
||||
#!/usr/bin/python3
|
||||
#
|
||||
# Copyright 2018 Ettus Research, a National Instruments Company
|
||||
#
|
||||
# SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
#
|
||||
# Description
|
||||
# Run the crossbar testbench (crossbar_tb) for varios parameter
|
||||
# configurations and generates load-latency graphs for each run.
|
||||
|
||||
import argparse
|
||||
import math
|
||||
import os, sys
|
||||
import shutil
|
||||
import glob
|
||||
import subprocess
|
||||
|
||||
g_tb_top_template = """
|
||||
`timescale 1ns/1ps
|
||||
module crossbar_tb_auto();
|
||||
crossbar_tb #(
|
||||
.TEST_NAME ("crossbar_tb_auto"),
|
||||
.ROUTER_IMPL ("{rtr_impl}"),
|
||||
.ROUTER_PORTS ({rtr_ports}),
|
||||
.ROUTER_DWIDTH ({rtr_width}),
|
||||
.MTU_LOG2 ({rtr_mtu}),
|
||||
.NUM_MASTERS ({rtr_sources}),
|
||||
.TEST_MAX_PACKETS ({tst_maxpkts}),
|
||||
.TEST_LPP ({tst_lpp}),
|
||||
.TEST_MIN_INJ_RATE ({tst_injrate_min}),
|
||||
.TEST_MAX_INJ_RATE ({tst_injrate_max}),
|
||||
.TEST_INJ_RATE_INCR (10),
|
||||
.TEST_GEN_LL_FILES (1)
|
||||
) impl (
|
||||
/* no IO */
|
||||
);
|
||||
endmodule
|
||||
"""
|
||||
|
||||
g_test_params = {
|
||||
'data': {'rtr_width':64, 'rtr_mtu':7, 'tst_maxpkts':100, 'tst_lpp':100, 'tst_injrate_min':30, 'tst_injrate_max':100},
|
||||
'ctrl': {'rtr_width':64, 'rtr_mtu':5, 'tst_maxpkts':100, 'tst_lpp':20, 'tst_injrate_min':10, 'tst_injrate_max':50},
|
||||
}
|
||||
|
||||
g_xb_types = {
|
||||
'chdr_crossbar_nxn':'data', 'axi_crossbar':'data',
|
||||
'axis_ctrl_2d_torus':'ctrl', 'axis_ctrl_2d_mesh':'ctrl'
|
||||
}
|
||||
|
||||
def get_options():
|
||||
parser = argparse.ArgumentParser(description='Run correctness sim and generate load-latency plots')
|
||||
parser.add_argument('--impl', type=str, default='chdr_crossbar_nxn', help='Implementation (CSV) [%s]'%(','.join(g_xb_types.keys())))
|
||||
parser.add_argument('--ports', type=str, default='16', help='Number of ports (CSV)')
|
||||
parser.add_argument('--sources', type=str, default='16', help='Number of active data sources (masters)')
|
||||
return parser.parse_args()
|
||||
|
||||
def launch_run(impl, ports, sources):
|
||||
run_name = '%s_ports%d_srcs%d'%(impl, ports, sources)
|
||||
# Prepare a transform map to autogenerate a TB file
|
||||
transform = {'rtr_impl':impl, 'rtr_ports':ports, 'rtr_sources':sources}
|
||||
for k,v in g_test_params[g_xb_types[impl]].items():
|
||||
transform[k] = v
|
||||
# Create crossbar_tb_auto.sv with specified parameters
|
||||
with open('crossbar_tb_auto.sv', 'w') as out_file:
|
||||
out_file.write(g_tb_top_template.format(**transform))
|
||||
# Create data directory for the simulation
|
||||
data_dir = os.path.join('data', impl)
|
||||
export_dir = os.path.join('data', run_name)
|
||||
try:
|
||||
os.makedirs('data')
|
||||
except FileExistsError:
|
||||
pass
|
||||
os.makedirs(data_dir)
|
||||
os.makedirs(export_dir)
|
||||
# Run "make xsim"
|
||||
exitcode = subprocess.Popen('make xsim TB_TOP_MODULE=crossbar_tb_auto', shell=True).wait()
|
||||
if exitcode != 0:
|
||||
raise RuntimeError('Error running "make xsim". Was setupenv.sh run?')
|
||||
# Generate load-latency graphs
|
||||
exitcode = subprocess.Popen('gen_load_latency_graph.py ' + data_dir, shell=True).wait()
|
||||
if exitcode != 0:
|
||||
raise RuntimeError('Error running "gen_load_latency_graph.py"')
|
||||
# Copy files
|
||||
os.rename('xsim.log', os.path.join(export_dir, 'xsim.log'))
|
||||
for file in glob.glob(os.path.join(data_dir, '*.png')):
|
||||
shutil.copy(file, export_dir)
|
||||
# Cleanup outputs
|
||||
subprocess.Popen('make cleanall', shell=True).wait()
|
||||
try:
|
||||
os.remove('crossbar_tb_auto.sv')
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
try:
|
||||
shutil.rmtree(data_dir)
|
||||
except OSError:
|
||||
print('WARNING: Could not delete ' + data_dir)
|
||||
|
||||
def main():
|
||||
args = get_options();
|
||||
for impl in args.impl.strip().split(','):
|
||||
for ports in args.ports.strip().split(','):
|
||||
for sources in args.sources.strip().split(','):
|
||||
launch_run(impl, int(ports), min(int(ports), int(sources)))
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in New Issue
Block a user