SC25: Confidence, Community, and Connections

The SC25 conference in St. Louis proved to be a transformative experience, taking me from nervous presenter to confident contributor in the HPC community. As someone whose research typically focuses on adversarial machine learning and network security, presenting work on Python environment management represented both a significant expansion of my skill set and an opportunity to contribute to a conversation about making HPC more accessible.

Making Connections: From Exhibition to Presentation

The exhibitions opened on the third day, and I explored several posters at the Texas A&M University booth. One that immediately stood out discussed building a Jupyter AI assistant for scientific workflows. With my software developer background, this project felt especially relevant. It wasn’t just about automation, it addressed a fundamental challenge: how do we empower domain scientists who aren’t HPC experts to leverage these powerful systems?

Later that week, I attended a talk from a researcher at ETH Zurich who discussed the future of HPC workloads and the growing need for dynamic, interactive scientific engines. The talk connected perfectly with the Jupyter AI work and highlighted how researchers with deep domain knowledge but limited HPC experience will increasingly rely on intuitive tools rather than traditional command-line workflows.

This theme became even more meaningful as I prepared to present my own research on ModuLair, our Python virtual environment management framework designed to simplify one of HPC’s most frustrating pain points.

Presenting ModuLair: Research Meets Reality

On workshop day, I had the opportunity to present “ModuLair: Streamlining Python Virtual Environment Management for HPC.” Standing before an audience of HPC professionals and researchers, I felt both the weight of representing months of collaborative work and the excitement of contributing to the conversation about HPC accessibility.

In my presentation, I explained how managing Python environments on HPC systems presents unique challenges. The manual multi-step process of navigating directories, loading compiler and Python modules in the correct order, creating environments, and configuring them for jobs is time-consuming and error-prone. For collaborative research, these challenges multiply as team members struggle to replicate each other’s environments across different sessions and systems.

What struck me most during the Q&A session was how many attendees nodded knowingly when I described these pain points. Several approached me afterward to share their own “environment hell” stories, tales of mismatched toolchains, mysterious import failures, and hours lost debugging ABI incompatibilities. It validated our research motivation in a deeply personal way.

Key Insights That Resonated

Reflective Design as a Guiding Principle

One of the most important insights from developing ModuLair was embracing reflective design, making system tradeoffs visible to users rather than hiding complexity. When we added Conda environment support, we deliberately included prompts that surface the storage implications and file count issues before creation. This approach respects user agency while promoting informed decision-making.

During my presentation, an attendee from Purdue mentioned their conda-env-mod tool, which takes a different approach by making Conda environments behave like standard HPC modules. Our discussion highlighted an interesting tension in HPC tool design: Do we adapt familiar tools like Conda to HPC constraints, or do we guide users toward HPC-native solutions like venv while preserving Conda as an escape hatch? ModuLair attempts the latter, and the validation from experienced HPC administrators suggested this philosophy resonates with the community’s evolving best practices.

Validation Through Real-World Usage

One aspect of our research that generated significant interest was our quantitative validation approach. Rather than relying on anecdotal evidence or download counts, we analyzed five months of actual usage data across three HPC clusters. The results were compelling: 82 unique users (9.8% of active Python users) adopted ModuLair within just five months, with 96.3% of adopters activating environments repeatedly, indicating real workflow integration rather than one-time trials.

Several attendees asked about our methodology for measuring “sustained adoption” versus “exploratory trials.” This led to a rich discussion about how we validate HPC tools more generally. Too often, our community measures success through citations or installation counts, but these metrics don’t capture whether tools actually improve daily research workflows. Our approach of combining Lmod audit logs with application-specific command logs provided a more complete picture.

A visualization researcher suggested we explore survival analysis techniques to better understand environment lifecycle patterns, which environments persist long-term versus being quickly abandoned. This kind of data could reveal what environment configurations are most stable and valuable to users, an idea I’m excited to pursue.

Broader Conversations: The Future of HPC

Beyond my presentation, several conference sessions deepened my understanding of where HPC is heading. The ETH Zurich talk on interactive scientific engines proved particularly thought-provoking. The speaker discussed how traditional batch-oriented HPC is being complemented by more interactive, exploratory workflows like Jupyter notebooks, visualization dashboards, and AI-assisted coding environments. ModuLair fits naturally into this transition by making environment setup nearly frictionless.

One recurring theme throughout the conference was the ongoing debate about Conda in HPC environments. My presentation’s discussion of Conda’s limitations, high file counts, metadata overhead, potential toolchain conflicts, resonated with many HPC administrators. Yet several researchers defended Conda as essential for their workflows, particularly in bioinformatics and data science where complex package dependencies are common.

This tension highlighted something important: there’s no one-size-fits-all solution for HPC Python environments. ModuLair’s strength lies not in prohibiting Conda, but in making the tradeoffs visible and providing a better default path while preserving Conda as an option when truly necessary. A computational biologist described their center’s hybrid approach using venv for most work but maintaining carefully managed Conda environments for genuinely complex situations. This aligns perfectly with ModuLair’s reflective design philosophy.

image.png

Community and Connection

On a fun side quest, the conference offered free professional headshots. This was my first time getting one taken, and it was a great opportunity to update my professional profile, something I hadn’t realized I needed until I started making connections throughout the week.

image.png

My roommates and I also attended several sponsored networking events. At the Dell/AMD reception, we even got caricature sketches of ourselves. At the Alien Appreciation Day reception, we explored the City Museum, a completely unique and whimsical architectural playground, and tried St. Louis’s famous toasted ravioli.

These social moments proved surprisingly valuable for research discussions. Some of my best conversations about ModuLair happened informally, over toasted ravioli, while waiting in line for caricatures, or during coffee breaks between sessions. I exchanged contact information with researchers from four different institutions who expressed interest in deploying ModuLair at their centers or collaborating on extensions.

One connection that particularly excites me: a researcher is interested in piloting ModuLair on their HPC system. This external deployment will provide valuable feedback on ModuLair’s portability and help us identify assumptions specific to our Texas A&M environment.

What I’m Taking Home

Several insights from SC25 will immediately impact my work. I’m implementing enhanced validation methodologies, including environment lifecycle tracking and correlation analysis between environment characteristics and longevity. The conversations about diverse user populations have inspired me to pursue integration with additional workflow composition tools and develop a REST API for easier third-party integration.

Beyond technical improvements, SC25 reinforced my belief that usability must be a first-class concern in HPC tool development. ModuLair’s adoption metrics demonstrate that investing in user experience, clear documentation, helpful error messages, GUI options, directly translates to tool uptake and sustained use. I plan to advocate for usability evaluation as a standard component of HPC tool development, not an afterthought.

The interest from other HPC centers has convinced me that ModuLair should evolve as a truly community-driven project. I’m working with our team to establish feedback mechanisms beyond our institution and create detailed deployment documentation for external adopters.

Finding My Rhythm

As a cybersecurity researcher whose work typically focuses on adversarial machine learning and network security, diving deep into HPC software engineering for ModuLair represented a significant expansion of my skill set. Presenting this work at SC25 pushed me further outside my comfort zone, but in the best possible way.

The HPC community’s warmth and intellectual generosity surprised me. Despite being relatively new to this subfield, I felt welcomed into conversations, taken seriously when I proposed ideas, and supported when I expressed uncertainty. This experience has reinforced my commitment to interdisciplinary research. The skills I’ve developed in reverse engineering and malware analysis have unexpected applications in understanding complex software systems, while the systems thinking required for ModuLair development enriches how I approach cybersecurity problems.

My TANGO@SC journey exceeded expectations in every dimension. I arrived hoping to learn about HPC trends and present our research competently. I left with a network of collaborators, a roadmap for ModuLair’s evolution, refined ideas about tool design and validation, and a genuine sense of belonging in the HPC community.

The conference reinforced that the future of HPC isn’t just about faster hardware or more efficient algorithms, it’s equally about making these powerful resources accessible to a broader community of researchers. Tools like ModuLair, AI-assisted workflows, and intuitive portals are all part of this democratization of HPC. Most importantly, I’ve found my rhythm in HPC: a balance between technical depth and accessibility, between individual innovation and community collaboration, between what HPC is today and what it can become.

To my TANGO cohort, our mentors, and everyone who made SC25 such a transformative experience: thank you. I’m already looking forward to SC26, where I hope to return not just as an attendee but as a more active contributor to the HPC community’s ongoing evolution.

Leave a Comment

Awesome! You've decided to leave a comment. Please keep in mind that comments are moderated.

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>