BenchCouncil Transactions on Benchmarks, Standards and Evaluations

Open access

ISSN: 2772-4859

BenchCouncil Transactions on Benchmarks, Standards and Evaluations

Open access

BenchCouncil Transactions on Benchmarks, Standards and Evaluations (TBench) publishes position articles that open new research areas; research articles that address new problems; methodologies; tools;...

BenchCouncil Transactions on Benchmarks, Standards and Evaluations (TBench) publishes position articles that open new research areas; research articles that address new problems; methodologies; tools; survey articles that build up comprehensive knowledge; and comment articles that argue published articles. Submissions should deal with benchmarks, standards and evaluation research areas. Particular areas of interest include, but are not limited to:

  1. Generalized benchmark science and engineering, for example:
    • Measurement standards
    • standardized data sets with defined properties
    • Representative workloads
    • Representative data sets
    • Best practices

  2. Benchmarks and standard specifications, implementations and validations of:
    • Big data
    • AI
    • HPC
    • Machine learning
    • Big scientific data
    • Data centers
    • Cloud
    • Warehouse-scale computing
    • Mobile robotics
    • Edge and fog computing
    • IoT
    • Blockchain
    • Data management and storage
    • Financial domains
    • Education domains
    • Medical domains
    • Other application domains

  3. Data sets
    • Detailed descriptions of research or industry datasets, including the methods used to collect the data and technical analyses supporting the quality of the measurements.
    • Analyses or meta-analyses of existing data and original articles on systems, technologies and techniques that advance data sharing and reuse to support reproducible research.
    • Articles evaluating the rigor and quality of the experiments used to generate the data and the completeness of the data description.
    • Tools that generate large-scale data while preserving their original characteristics.

  4. Workload characterization, quantitative measurement, design and evaluation studies of:
    • Computer and communication networks, protocols and algorithms
    • Wireless, mobile, ad-hoc and sensor networks, and IoT applications
    • Computer architectures, hardware accelerators, multi-core processors, memory systems and storage networks
    • High-performance computing
    • Operating systems, file systems and databases
    • Virtualization, data centers, distributed and cloud computing, fog and edge computing
    • Mobile and personal computing systems
    • Energy-efficient computing systems
    • Real-time and fault-tolerant systems
    • Security and privacy of computing and networked systems
    • Software systems and services, and enterprise applications
    • Social networks and multimedia systems, web services
    • Cyber-physical systems, including the smart grid

  5. Methodologies, metrics, abstractions, algorithms and tools for:
    • Analytical modelling techniques and model validation
    • Workload characterization and benchmarking
    • Performance, scalability, power and reliability analysis
    • Sustainability analysis and power management
    • System measurement, performance monitoring and forecasting
    • Anomaly detection, problem diagnosis and troubleshooting
    • Capacity planning, resource allocation, run time management and scheduling
    • Experimental design, statistical analysis and simulation

  6. Measurement and evaluation:
    • Evaluation methodology and metrics
    • Testbed methodologies and systems
    • Instrumentation, sampling, tracing, and profiling of large-scale, real-world applications and systems
    • Collection and analysis of measurement data that yield new insights
    • Measurement-based modelling (e.g., workloads, scaling behavior and assessment of performance bottlenecks)
    • Methods and tools to monitor and visualize measurement and evaluation data
    • Systems and algorithms that build on measurement-based findings
    • Advances in data collection, analysis, and storage (e.g., anonymization, querying and sharing)
    • Reappraisals of previous empirical measurements and measurement-based conclusions
    • Descriptions of challenges and future directions that the measurement and evaluation community should pursue

Society affiliation

International Open Benchmark Council (BenchCouncil) is a non-profit organization that aims to benchmark, standardize, evaluate and incubate emerging technologies. Since its founding, BenchCouncil bears four fundamental responsibilities: establish unified benchmark science and engineering across multi-disciplines; define the problems or challenges ...

International Open Benchmark Council (BenchCouncil) is a non-profit organization that aims to benchmark, standardize, evaluate and incubate emerging technologies. Since its founding, BenchCouncil bears four fundamental responsibilities: establish unified benchmark science and engineering across multi-disciplines; define the problems or challenges of emerging and future computing; keep the benchmarks and standards community open, inclusive, and growing; and promote benchmark-based quantitative approaches to tackle multidisciplinary and interdisciplinary challenges. BenchCouncil also hosts a series of influential benchmark projects. BenchCouncil presents the achievement and rising star awards each year at its flagship Bench conference.

News

BenchCouncil Transactions on Benchmarks, Standards and Evaluations is now Indexed in DOAJ

BenchCouncil Transactions on Benchmarks, Standards and Evaluations is Open for Submissions

View all

Related journals

Stay Informed

Register your interest and receive email alerts tailored to your needs. Sign up below.