BenchCouncil Transactions on Benchmarks, Standards and Evaluations
ISSN: 2772-4859
BenchCouncil Transactions on Benchmarks, Standards and Evaluations
BenchCouncil Transactions on Benchmarks, Standards and Evaluations (TBench) publishes position articles that open new research areas; research articles that address new problems; methodologies; tools;...
BenchCouncil Transactions on Benchmarks, Standards and Evaluations (TBench) publishes position articles that open new research areas; research articles that address new problems; methodologies; tools; survey articles that build up comprehensive knowledge; and comment articles that argue published articles. Submissions should deal with benchmarks, standards and evaluation research areas. Particular areas of interest include, but are not limited to:
- Generalized benchmark science and engineering, for example:
- Measurement standards
- standardized data sets with defined properties
- Representative workloads
- Representative data sets
- Best practices
- Benchmarks and standard specifications, implementations and validations of:
- Big data
- AI
- HPC
- Machine learning
- Big scientific data
- Data centers
- Cloud
- Warehouse-scale computing
- Mobile robotics
- Edge and fog computing
- IoT
- Blockchain
- Data management and storage
- Financial domains
- Education domains
- Medical domains
- Other application domains
- Data sets
- Detailed descriptions of research or industry datasets, including the methods used to collect the data and technical analyses supporting the quality of the measurements.
- Analyses or meta-analyses of existing data and original articles on systems, technologies and techniques that advance data sharing and reuse to support reproducible research.
- Articles evaluating the rigor and quality of the experiments used to generate the data and the completeness of the data description.
- Tools that generate large-scale data while preserving their original characteristics.
- Workload characterization, quantitative measurement, design and evaluation studies of:
- Computer and communication networks, protocols and algorithms
- Wireless, mobile, ad-hoc and sensor networks, and IoT applications
- Computer architectures, hardware accelerators, multi-core processors, memory systems and storage networks
- High-performance computing
- Operating systems, file systems and databases
- Virtualization, data centers, distributed and cloud computing, fog and edge computing
- Mobile and personal computing systems
- Energy-efficient computing systems
- Real-time and fault-tolerant systems
- Security and privacy of computing and networked systems
- Software systems and services, and enterprise applications
- Social networks and multimedia systems, web services
- Cyber-physical systems, including the smart grid
- Methodologies, metrics, abstractions, algorithms and tools for:
- Analytical modelling techniques and model validation
- Workload characterization and benchmarking
- Performance, scalability, power and reliability analysis
- Sustainability analysis and power management
- System measurement, performance monitoring and forecasting
- Anomaly detection, problem diagnosis and troubleshooting
- Capacity planning, resource allocation, run time management and scheduling
- Experimental design, statistical analysis and simulation
- Measurement and evaluation:
- Evaluation methodology and metrics
- Testbed methodologies and systems
- Instrumentation, sampling, tracing, and profiling of large-scale, real-world applications and systems
- Collection and analysis of measurement data that yield new insights
- Measurement-based modelling (e.g., workloads, scaling behavior and assessment of performance bottlenecks)
- Methods and tools to monitor and visualize measurement and evaluation data
- Systems and algorithms that build on measurement-based findings
- Advances in data collection, analysis, and storage (e.g., anonymization, querying and sharing)
- Reappraisals of previous empirical measurements and measurement-based conclusions
- Descriptions of challenges and future directions that the measurement and evaluation community should pursue
Society affiliation
International Open Benchmark Council (BenchCouncil) is a non-profit organization that aims to benchmark, standardize, evaluate and incubate emerging technologies. Since its founding, BenchCouncil bears four fundamental responsibilities: establish unified benchmark science and engineering across multi-disciplines; define the problems or challenges ...
International Open Benchmark Council (BenchCouncil) is a non-profit organization that aims to benchmark, standardize, evaluate and incubate emerging technologies. Since its founding, BenchCouncil bears four fundamental responsibilities: establish unified benchmark science and engineering across multi-disciplines; define the problems or challenges of emerging and future computing; keep the benchmarks and standards community open, inclusive, and growing; and promote benchmark-based quantitative approaches to tackle multidisciplinary and interdisciplinary challenges. BenchCouncil also hosts a series of influential benchmark projects. BenchCouncil presents the achievement and rising star awards each year at its flagship Bench conference.
