run_tetwild_sweep Namespace Reference

Functions
	_kill (popen)
	_handle_signal (signum, _frame)
	_mem_wrapper (model_id)
	_memory_capped_available ()
	already_done (model_id)
	run_one (mesh_path)
	_describe_exit (code, work)
	_prune (work)
	_read_status (d)
	_parse_success (d)
	_histogram (title, values, edges, fmt="{:g}")
	_esc (s)
	_fmt (v, nd=2)
	_bucket (values, edges)
	_bars (labels, counts, unit, alt=False)
	_html_hist (values, edges, unit)
	_stat (k, v, u="", cls="")
	_table (headers, rows, reason_col=None, empty="(none)")
	generate_html_report (rows, stats, report_dir, out_dir, dataset_dir=None, mesh_extensions=())
	generate_report ()
	_reason_bucket (reason)
	_order_models (meshes)
	run_sweep ()
	stop_sweep ()

Variables
	SWEEP_ROOT = Path("/u/3/daniele/thingi10k-sweep")
str	WMTK_APP = SWEEP_ROOT / "build/app/wmtk_app"
str	DATASET_DIR = SWEEP_ROOT / "data"
	OUT_DIR = Path(os.environ.get("TETWILD_OUT", str(SWEEP_ROOT / "runs/full")))
tuple	MESH_EXTENSIONS = ("*.stl", "*.obj", "*.ply", "*.off")
	PARALLEL = max(1, int(os.environ.get("TETWILD_PARALLEL", "8")))
	THREADS = max(1, int(os.environ.get("TETWILD_THREADS", "8")))
	JOB_TIMEOUT = int(os.environ.get("TETWILD_JOB_TIMEOUT", "10800"))
	MEM_GB = int(os.environ.get("TETWILD_MEM_GB", "128"))
	LIMIT = int(os.environ.get("TETWILD_LIMIT", "0"))
	SAMPLE = os.environ.get("TETWILD_SAMPLE", "smallest")
	SEED = int(os.environ.get("TETWILD_SEED", "0"))
	REPORT_ONLY = bool(os.environ.get("TETWILD_REPORT_ONLY"))
dict	PARAMS
list	KEEP_GLOBS = ["*.msh", "*.obj", "*.log", "*.json", "status.txt"]
str	SUCCESS_DIR = OUT_DIR / "success"
str	FAILURE_DIR = OUT_DIR / "failure"
str	WORK_DIR = OUT_DIR / ".work"
str	REPORT_DIR = OUT_DIR / "report"
str	PIDFILE = OUT_DIR / "sweep.pid"
	_STOP = threading.Event()
dict	_children = {}
	_children_lock = threading.Lock()
	IT_RE = re.compile(r"========it (\d+)========")
	PHASES_RE
str	_CSS
str	_JS

Detailed Description

Batch-run tetwild over the Thingi10K dataset -- kirby.cs.nyu.edu edition.

Adapted from run_tetwild_thingi10k.py (the macOS version). Same contract: run it
with no arguments to start (or resume) the sweep; it skips every model already in
success/ or failure/. Stop it cleanly with:

    run_tetwild_thingi10k_kirby.py stop

which signals the running sweep to abandon whatever is in flight (those models are
left unpublished, so a later resume reprocesses them -- no spurious failures), drain,
write the report, and exit.

What changed from the macOS version:
  * paths point at /u/3/daniele/thingi10k-sweep;
  * OUT_DIR is env-overridable (TETWILD_OUT), so a trial sweep and the real one do
    not share a success/failure namespace;
  * defaults are 8 models x 8 threads, 3h per model;
  * each model runs inside a systemd transient scope with a hard memory cap, so a
    runaway model is OOM-killed instead of taking the machine down with it (kirby is
    shared); see the memory note below;
  * TETWILD_SAMPLE picks *which* models a truncated run gets -- 'smallest' is the
    original behaviour, 'spread' covers the whole size range, which is what you want
    from a 100-model trial;
  * the report tells OOM kills apart from other signals.

Per model:
  * write a tetwild JSON (eps_rel 1e-3, filter none);
  * run wmtk_app in a scratch dir under the output volume, capped in time and memory;
  * exit 0                    -> move the run into  <OUT>/success/<id>/
    nonzero / timeout / OOM   -> move the run into  <OUT>/failure/<id>/  (with a reason)

Configuration -- environment variables:
    TETWILD_OUT           output directory            (default runs/full)
    TETWILD_PARALLEL      models to run concurrently  (default 8)
    TETWILD_THREADS       threads per model           (default 8)
    TETWILD_JOB_TIMEOUT   per-model seconds           (default 10800 = 3h)
    TETWILD_MEM_GB        per-model memory cap, GB    (default 128, 0 disables)
    TETWILD_LIMIT         process at most N new models (default 0 = all)
    TETWILD_SAMPLE        name | smallest | spread | random  (default smallest)
    TETWILD_SEED          seed for TETWILD_SAMPLE=random  (default 0)
    TETWILD_REPORT_ONLY   if set, only regenerate the report and exit

Memory note: the cap is PER MODEL, not a budget for the sweep. 8 x 128G is more than
kirby has, so the cap is a runaway-killer, not an admission control -- it stops one
pathological mesh from swapping the box, and does nothing in the normal case. Lower
TETWILD_MEM_GB if the sweep has to coexist with someone else's job.

Function Documentation

_describe_exit()

run_tetwild_sweep._describe_exit (   code,   work  )

protected

Human-readable reason for a nonzero exit, distinguishing the OOM kill.

A cgroup OOM kill arrives as SIGKILL, which is also what a manual kill looks
like -- but we only kill on stop or timeout, and both are handled before this
is reached, so an unexplained SIGKILL under a cap is the OOM killer. systemd
says so on the scope's stderr when it can, so prefer that when it is there.

_histogram()

run_tetwild_sweep._histogram (   title,   values,   edges,   fmt = "{:g}"  )

protected

ASCII histogram. edges is a list of bucket boundaries; last bucket is open.

_memory_capped_available()

run_tetwild_sweep._memory_capped_available ( )

protected

Probe once, so a broken systemd user manager fails loudly at startup.

_order_models()

run_tetwild_sweep._order_models (   meshes )

protected

Order the work list. Only matters when the run is truncated (LIMIT / a stop).

smallest: cheapest first, so a partial run covers as many models as possible and
          the timeout mostly bites the giant meshes, which come last. This is the
          right order for the full sweep.
spread:   walk the size-sorted list at a stride, so a truncated run samples the
          whole size range. A 100-model trial ordered 'smallest' tells you almost
          nothing -- the 100 smallest meshes in Thingi10K are trivial.
random:   uniform sample, seeded by TETWILD_SEED.
name:     filename order. Size is uncorrelated with the name, so the expensive
          models are spread through the run instead of all landing at the end --
          which keeps the machine busy to the finish and makes progress linear
          rather than front-loaded. This is what the 2D runner defaults to.

_parse_success()

run_tetwild_sweep._parse_success (   d )

protected

time, iterations, energies for one successful model.

generate_html_report()

run_tetwild_sweep.generate_html_report	(		rows,
			stats,
			report_dir,
			out_dir,
			dataset_dir = None,
			mesh_extensions = ()
	)

Write report/index.html. `rows` are the same dicts the CSV is built from.

run_one()

run_tetwild_sweep.run_one

(

mesh_path

)

Run tetwild on one mesh. Returns 'success' / 'failure' / 'stopped'.

stop_sweep()

run_tetwild_sweep.stop_sweep

(

)

Signal a running sweep to stop cleanly (via its pidfile).

Variable Documentation

_JS

str run_tetwild_sweep._JS

protected

Initial value:

=  """
(function () {
  var tip = document.getElementById('tip');
  document.querySelectorAll('[data-tip]').forEach(function (el) {
    el.addEventListener('mousemove', function (e) {
      tip.textContent = el.getAttribute('data-tip');
      tip.style.opacity = '1';
      var x = e.clientX + 14, y = e.clientY + 16;
      var w = tip.offsetWidth, h = tip.offsetHeight;
      if (x + w > window.innerWidth - 8) x = e.clientX - w - 14;
      if (y + h > window.innerHeight - 8) y = e.clientY - h - 16;
      tip.style.left = x + 'px'; tip.style.top = y + 'px';
    });
    el.addEventListener('mouseleave', function () { tip.style.opacity = '0'; });
  });
})();
"""

PARAMS

dict run_tetwild_sweep.PARAMS

Initial value:

=  {
    "application": "tetwild",
    "eps_rel": 1e-3,
    "filter": "none",
    "num_threads": THREADS,
    # The .msh is the real output and the .vtu is a visualization dump that _prune
    # deletes immediately afterwards, so writing it costs time and disk on every model
    # for nothing. The 2D sweep has always forced this off.
    "write_vtu": False,
}

PHASES_RE

run_tetwild_sweep.PHASES_RE

Initial value:

=  re.compile(
    r"TETWILD_PHASES threads (\S+) load (\S+)s simplify (\S+)s insertion (\S+)s "
    r"optimization (\S+)s finalize (\S+)s output (\S+)s")