BenchCouncil Transactions on Benchmarks, Standards and Evaluations (TBench) is an open-access journal dedicated to advancing the field of benchmarks, data sets, standards, evaluations and optimizations.

We invite submissions covering a wide range of topics from various disciplines, with a particular emphasis on interdisciplinary research. Whether it pertains to computers, AI, medicine, education, finance, business, psychology, or other social disciplines, all relevant contributions are welcome.

At TBench, we prioritize the reproducibility of research. We strongly encourage authors to ensure that their articles are prepared for open-source or artifact evaluation before submission.

Areas of interest include, but are not limited to:

1. Problem definition as a benchmark
• Position articles on the definitions of emerging or future problems/ challenges
• Position articles on opening new research areas
• Position articles on investigating the impact of new technologies on different disciplines
• Research articles on instantiating new problem settings
2. Evaluation standard as a benchmark
• Definition, design, implementation, and validation of evaluation standards
• Evaluation methodology and metrics
• New abstractions and tools in evaluation
• Simulation, emulation, and testbed methodologies and systems in evaluation
3. Industry best practice as a benchmark
• Searching and summarizing industry best practice
• Evaluation and optimization of industry practice
• Retrospective of industry practice
• Characterizing and optimizing real-world applications and systems
• Benchmarking
4. State-of-the-art solution as a benchmark
• State-of-the-art solutions to well-known benchmarks
• Benchmarking state-of-the-art solutions
• Preliminary but insightful solutions to new or emerging problems or benchmarks
• Evaluations of state-of-the-art solutions in the real-world setting
5. Data set as a benchmark
• Explicit or implicit problem definition deduced from the data set
• Detailed descriptions of research or industry datasets, including the methods used to collect the data and technical analyses supporting the quality of the measurements
• Analyses or meta-analyses of existing data
• Systems, technologies, and techniques that advance data sharing and reuse to support reproducible research
• Tools that generate large-scale data while preserving their original characteristics
• Evaluating the rigor and quality of the experiments used to generate the data and the completeness of the data description
6. Workload characterization, evaluation, and retrospective of design/implementation of real-world:
• Computer or AI applications or systems
• Finance applications or systems
• Education applications or systems
• Business applications or systems
• Medicine applications or systems
• Other industry applications or systems
7. Measurement and evaluation:
• Instrumentation, sampling, tracing, and profiling of large-scale, real-world applications and systems
• Collection and analysis of measurement data that yield new insights
• Measurement-based modeling (e.g., workloads, scaling behavior, and assessment of performance bottlenecks)
• Methods and tools to monitor and visualize measurement and evaluation data
• Systems and algorithms that build on measurement-based findings
• Advances in data collection, analysis, and storage, e.g., anonymization, querying, and sharing
• Reappraisal of previous empirical measurements and measurement-based conclusions
• Descriptions of challenges and future directions that the measurement and evaluation community should pursue
8. Methodologies, metrics, abstractions, algorithms, and tools for:
• Analytical modeling techniques and model validation
• Workload characterization and benchmarking
• Performance, scalability, power, and reliability analysis
• Sustainability analysis and power management
• System measurement, performance monitoring, and forecasting
• Anomaly detection, problem diagnosis, and troubleshooting
• Capacity planning, resource allocation, run-time management, and scheduling
• Experimental design, statistical analysis, and simulation