Namespace Idetifier Mapping for Splunk Analysis of Supercomputer Data Logs
Mentor:Olaf Faaland, System Administrator in HPC at Lawrence Livermore National Laboratory, Lawrence Livermore National Laboratory (LLNL)
Lawrence Livermore National Laboratory (LLNL) has one of the most powerful computing facilities in the world with dozens and dozens of massive systems including two of the ten fastest computers in the world. Due to the amount of hardware and software even a linear scale on the size of the system to the number of problems that arise means that system information and error logs can number in the billions and, for longer time periods, the trillions. To handle the ever growing volume of information Livermore Computation is beginning to use Splunk, a software for searching, sorting, and analyzing big data. Splunk uses the Search Processing Language (SPL) a unique search language derived from SQL and Unix piping. My project over the summer involved writing scripts to give Splunk useful information that was specific to LLNL resources and researching new ways for Splunk to identify and locate events that indicate issues such as communication failures, excessive log in attempts, or hardware failures.