Documentation

Introduction

What is Evalda?

Evalda is the automated judge platform for the DataQuest hackathon, organized by the IEEE INSAT SB Computer Society Chapter and ACM INSAT. You train a machine learning model, submit it, and Evalda runs it against a hidden validation set — returning your score in real time.

Competition Format

DataQuest is a 10-hour, 2-phase hackathon.

Quick Start

From sign-up to your first score in 5 steps.

1. 1

   Register

   Head to the sign-up page and register with the same email you used in the registration form. Your team is pre-assigned — you’ll be placed automatically.
2. 2

   Check your team

   Go to the Team Dashboard and confirm your team and teammates are correct. If something’s off, contact the organizers immediately.
3. 3

   Train your model

   Download the dataset, explore, and train your model using any Python ML framework you want. Then fill in the 3 functions in solution.py (see template below).
4. 4

   Package & submit

   Zip exactly 3 files: solution.py, model.*, and requirements.txt. Upload the .zip on the Submit page.
5. 5

   Get your score

   Evalda installs your dependencies, runs your code against the hidden validation set, and streams your score back in real time. Track all attempts on the Team Dashboard.

Platform Overview

Evalda has four main areas, all accessible from the navigation bar.

Phase 1: Build Your Model

The Dataset

The solution.py Template

Your submission must include a solution.py file with exactly these 3 functions. Do not change the function signatures.

1# Import your libraries here
2 
3def preprocess(df):
4    # Preprocess the raw input DataFrame.
5    # Receives the test data WITHOUT the label column.
6    return df
7 
8def load_model():
9    # Load and return your trained model from disk.
10    # Example: return joblib.load('model.pkl')
11    model = None
12    return model
13 
14def predict(df, model):
15    # Generate predictions on the preprocessed DataFrame.
16    # This is the only function that is timed.
17    predictions = None
18    return predictions

Evalda calls these functions in order: preprocess → load_model → predict. Only predict is timed for the latency metric.

The Environment

Your code runs inside a Python 3.10 container with the following packages pre-installed:

Packaging Your Submission

Your .zip must contain exactly these 3 files at the root level (no nested folders):

Scoring System

Your final score is a composite of three factors: prediction accuracy, model file size, and inference latency.

score = accuracy × size_penalty × duration_penalty
 
where:
  accuracy         = correct_predictions / total_predictions
  size_penalty     = max(0.5, 1 − model_size_mb / 100)
  duration_penalty = max(0.5, 1 − predict_duration_seconds / 10)

Both penalties floor at 0.5, so even a huge or slow model can’t lose more than half its accuracy score. That said, a smaller and faster model will rank higher at equal accuracy.

Metrics Breakdown

Submission Limits

Each team has 20 submission attempts for the entire competition. Only accepted submissions count toward this limit — rejected ones are free.

Your remaining attempts are always visible on the Submit page.

Leaderboard

The leaderboard is anonymous by design — team names are hidden from everyone. You can only see your own team’s name on the board. Only your best accepted score is shown.

Phase 2: Build Your App

In the final 3 hours, your goal shifts from model training to building a complete application around your model.

Your Team

Teams are pre-assigned based on the registration form you filled out before the competition. When you sign up on Evalda with the same email, you’ll be placed into your team automatically.

There is nothing to create or join. Just register and verify on the Team Dashboard that everything looks correct. If there’s an issue, contact the organizers immediately.

Competition Rules

By participating you agree to the following. Violations may result in disqualification.

• —Teams are exactly 4 members.
• —Each participant belongs to exactly one team.
• —All submissions must be the team’s own original work.
• —Sharing code, model files, or solutions between teams is strictly prohibited.
• —Probing the auto-judger to reverse-engineer the hidden validation set is not allowed.
• —Any attempt to exploit or circumvent the submission system results in immediate disqualification.
• —Use of publicly available datasets and pre-trained models strictly prohibited.
• —Each team has 20 accepted submission attempts. Use them wisely.

Academic Integrity

DataQuest is an academic competition. Submitted solutions must be designed and implemented by your team. Plagiarism — including copying solutions from other teams or prior competitions — is grounds for disqualification.

Allowed Model Formats

Your model file must be named model.<ext> using one of the following extensions:

Submission Constraints