RBoot: Software Infrastructure for a Remote FPGA Laboratory

Kushal Datta and Ron Sass

Traditional FPGA education either involves a physical laboratory room with workstations connected to individual FPGA experimenter boards or simulation platforms. Physical labs are expensive to maintain and require substantial floor space. In addition, students need to be physically present in the laboratories to access the FPGA boards. On the other hand, it is often the case that simulation platforms do not provide the students an adequate, in depth understanding of concepts (such as synthesis on FPGAs). In this short paper, a third option – a remote FPGA Laboratory – is illustrated. This option uses software to allow students and researchers in geographically distributed locations to remotely access a pool of 64XilinxML-310 Boards over the Internet. To the best of our knowledge, a remote FPGA facility of this kind is unique. The infrastructure described here gives remote users the ability to dynamically power on/off the FPGA boards, upload/download files, configure the boards online and execute synthesized designs while sending input and output through the Internet. Advantages include considerable savings in space, greater accessibility for users, ease of maintenance, and the ability to better schedule resources.

FPGA laboratories are usually setup with individual FPGA boards connected to workstations through USB, PCI or JTAG cable connections. The modus operandiis students have to reserve the lab stations and physically be present to run their designs. Although, physical labs enhance students' interests in scientific learning and provides them with hands-on experience [3], such a facility has a number of drawbacks. First, housing multiple workstation and FPGA experimenter boards require a significant amount of laboratory space – a valuable resource at many educational institutions. Second, system administrators and lab assistants have to be employed to monitor and log activities to ensure safety and security of the expensive equipment. Frequently, stations are manually updated which, in large labs, can lead to software inconsistencies. Third, it is necessary to maintain schedules or reserve times that the lab is physically available to students and ensure that every user gets adequate time with the limited FPGA resources. The problem of scheduling becomes harder in presence of a large group of users. On the other hand, simulation platforms makes it easier to maintain laboratories (because they can piggy-back on other resources) and do not require specialized equipment, reducingcosts.

Simulated labs are also effective in the sense that any simulated step can be stopped at any moment to review and get a better understanding of the system [5]. However, simulations shadow the real execution parameters in a virtual world [4]. Realistic simulations can take long time to execute and still may fail to precisely model the real parameters and their effects. Moreover, costs involved in maintaining and running simulation software is not always less compared to experimental hardware apparatus.

The general idea is to make a pool of FPGA resources accessible to users over the Internet. Currently, users log in to a server with ssh and use client on the server to establish a session with a FPGA board. The session software has to keep track of what boards are in use (to prevent users from accidentally overwriting each others' work), provide a facility for controlling the boards remotely and handle I/O to the board. In this section, we briefly describe our lab setup, the fpga-session software, and a remote boot and control client/server software.

The physical setup includes two standard vertical 19" racks that house sixty-four 1UML-310(FPGA) boards. Remaining space in the racks are used to hold the main server and networking (currently an HP Procurve 10/100 Ethernet switch), and a firewall. Inside the racks there are three 24port power distribution units (PDU) used to power to the ML-310 boards. These are switched PDUs which allows the main server to turned on or off individual outlets. (This is accomplished via the Ethernet network using Simple Network Management Protocol (SNMP) [1].) The logical organization of the remote lab is shown in Figure 1.

Figure 1. Organization and Interfaces of the FPGA Boards

The consoles of all sixty-four ML-310 boards are concentrated on a USB network and connected to the main server. Each ML-310 board is also connected to the Ethernet switch and use DHCP to get their assigned IP address. Ethernet is used to carry most of the data, including FPGA bitstreams. ML-310 boards come with a Compact Flashcard and we replace the default Power-On Self Test (POST) application with our own rbootctld software (described below). The fpga-session program runs on the main server. Every user must first run this command to allocate a board. The application uses three files /etc/boards, /etc/users,and /var/inuse which hold system configuration,valid users,and current state information,respectively.

It is evident that in such a remote lab model, there might be concurrent accesses to the fpga-session program from different users. To counteract,alock-file mechanism is used to establish mutual exclusion. The inuse file is locked until the connection to the allocated board is successfully established for a user and the change in its status is reflected back in the inuse file. When logging out, the user needs to wait for other users to release the lock on the file, if any, and complete their startup protocol. However, there is no chance of deadlock as the locks on files are released as soon as the status updates are completed. The last component of the infrastructure is the remote boot control program. This consists of a client program rbootctl that runs on the main server station and our replacement for the POST, rbootctld. The latter does the normal power-on self test and then boots a simple,dedicated Linux system with an Ethernet and SysACE driver. A program is launched on the console that is an enhanced version of the one provided by Xilinx and allows interactive users to boot to an arbitrary configuration slot. It also launches a daemon that allows remote users to transfer bitstreams, configure the SysACE, and boot an arbitrary configuration slot. In addition to rbootctl, there is another program on the main server, fish, which is used to do a hard reset any ML310 board (using the SNMP to interact with PDU).

Several instances of web-based FPGA accesses are developed and designed. In [2] the authors integrated their custom bio informatics sequence-matching hardware accelerator with the university-wide grid to aid biology students use the FPGA boards. In most of the above cases, FPGA resources are available as custom hardware platforms. To the best of our knowledge, RemoteBoot is the first time FPGA boards are available over the Internet for experimental purposes.

In this paper, we have proposed a novel solution to improve student learning with limited resources. As discussed in the Section 2, current state-of-the-art FPGA education uses either traditional hands-on laboratories or expensive licensed version simulation platforms. The software infrastructure described here introduces a third option: remote labs. The advantages of this approach include, minimal space, greater accessibility, easier maintanence and lower management cost. On the contrary, while remote labs have the potential to provide affordable real experimental data through sharing experimental devices, their educational effectiveness might be questioned. Teachers argue that some students feel a loss of education from not being able to physical touch the hardware. In cases of complicated designs, debugging might be a issue where observing useful LEDs may provide a easier respite. Although there are alleys to transmit back debug data into our system using traditional connections and protocols, this aspect of hardware design is not quite handled in this work. We wish to continue this work and address such issues in our future work. Also, remote labs might not be a suitable option in hands-on labs where students add breadboards as physical components.

1. J. Case, M. Fedor, M. Schoffstall, and J. Davin. Rfc 1157 simple network management protocol (snmp).
2. R. K. Karanam, A. Ravindran, A. Mukherjee, C. Gibas, and A. B. Wilkinson. Using fpga-based hybrid computers for bioinformatics applications. Xcell Journal, 2006.
3. J. Ma and J. V. Nickerson. Hands-on, simulated, and remote laboratories: A comparative literature review. ACM Comput. Surv., 38(3):7, 2006.
4. D. J. Magin and S. Kanapathipillai. Engineering students understanding of the role of experimentation. European J. Eng. Education, 25(4):351–358, 2000.
5. A. Parush, H. Hamm, and A. Shtub. Learning histories in simulation-based teaching: The effects on self-learning and transfer. Comput. and Education, 39:319-332, 2002.