run_simwild_sweep Namespace Reference

Functions
	_kill (popen)
	_handle_signal (signum, _frame)
	_mem_wrapper (model_id)
	_memory_capped_available ()
	already_done (model_id)
	run_one (mesh_path)
	_describe_exit (code, work)
	_prune (work)
	_read_status (d)
	_parse_success (d)
	_histogram (title, values, edges, fmt="{:g}")
	_esc (s)
	_fmt (v, nd=2)
	_bucket (values, edges)
	_bars (labels, counts, unit, alt=False)
	_html_hist (values, edges, unit)
	_stat (k, v, u="", cls="")
	_table (headers, rows, reason_col=None, empty="(none)")
	generate_html_report (rows, stats, report_dir, out_dir, dataset_dir=None, mesh_extensions=())
	generate_report ()
	_reason_bucket (reason)
	_order_models (meshes)
	run_sweep ()
	stop_sweep ()

Variables
	SWEEP_ROOT = Path(os.environ.get("SIMWILD_ROOT", "/u/3/daniele/simwild-thingi10k-sweep"))
str	WMTK_APP = SWEEP_ROOT / "build/app/wmtk_app"
str	DATASET_DIR = SWEEP_ROOT / "data"
	OUT_DIR = Path(os.environ.get("SIMWILD_OUT", str(SWEEP_ROOT / "runs/full")))
tuple	MESH_EXTENSIONS = ("*.stl", "*.obj", "*.ply", "*.off")
	PARALLEL = max(1, int(os.environ.get("SIMWILD_PARALLEL", "8")))
	THREADS = max(1, int(os.environ.get("SIMWILD_THREADS", "8")))
	JOB_TIMEOUT = int(os.environ.get("SIMWILD_JOB_TIMEOUT", "10800"))
	MEM_GB = int(os.environ.get("SIMWILD_MEM_GB", "128"))
	LIMIT = int(os.environ.get("SIMWILD_LIMIT", "0"))
	SAMPLE = os.environ.get("SIMWILD_SAMPLE", "smallest")
	SEED = int(os.environ.get("SIMWILD_SEED", "0"))
	REPORT_ONLY = bool(os.environ.get("SIMWILD_REPORT_ONLY"))
dict	PARAMS
list	KEEP_GLOBS = ["*.log", "*.json", "status.txt"]
str	SUCCESS_DIR = OUT_DIR / "success"
str	FAILURE_DIR = OUT_DIR / "failure"
str	WORK_DIR = OUT_DIR / ".work"
str	REPORT_DIR = OUT_DIR / "report"
str	PIDFILE = OUT_DIR / "sweep.pid"
	_STOP = threading.Event()
dict	_children = {}
	_children_lock = threading.Lock()
	IT_RE = re.compile(r"========it (\d+)========")
str	_CSS
str	_JS

Detailed Description

Batch-run simwild over the Thingi10K dataset -- kirby.cs.nyu.edu edition.

Adapted from tetwild's run_tetwild_sweep.py: same contract, environment layout and
output shape (only the TETWILD_* variables become SIMWILD_*), so a simwild sweep and
a tetwild sweep can run side by side on the same machine without either one's settings
leaking into the other. Run it with no arguments to start (or resume) the sweep; it
skips every model already in success/ or failure/. Stop it cleanly with:

    run_simwild_sweep.py stop

which signals the running sweep to abandon whatever is in flight (those models are
left unpublished, so a later resume reprocesses them -- no spurious failures), drain,
write the report, and exit.

Each model is a single closed surface mesh, fed through simwild with
tag_from_winding_number left at its default (true): the arrangement's one cell is
tagged inside/outside by winding number, so no curve network or pre-tagged .msh is
needed -- the same single-input shape tetwild's Thingi10K sweep uses.

What changed from tetwild's version:
  * application is "simwild", not "tetwild";
  * simwild has no `filter` knob (that is tetwild's post-insertion outside-removal
    pass; simwild's single-input case is tagged by winding number instead), so it is
    dropped from PARAMS rather than pinned;
  * `eps_rel` is pinned at simwild's own spec default (2e-3), not tetwild's (1e-3) --
    the two applications do not share a default;
  * paths point at /u/3/daniele/simwild-thingi10k-sweep, a separate root from
    tetwild's sweep so the two never share a success/failure namespace;
  * SIMWILD_SAMPLE picks *which* models a truncated run gets -- 'smallest' is the
    original behaviour, 'spread' covers the whole size range, which is what you want
    from a 100-model trial;
  * the report tells OOM kills apart from other signals.

Per model:
  * write a simwild JSON (eps_rel 2e-3, single-input winding-number tagging);
  * run wmtk_app in a scratch dir under the output volume, capped in time and memory;
  * exit 0                    -> move the run into  <OUT>/success/<id>/
    nonzero / timeout / OOM   -> move the run into  <OUT>/failure/<id>/  (with a reason)

Configuration -- environment variables:
    SIMWILD_ROOT          sweep root: build/ data/ runs/ live here
    SIMWILD_OUT           output directory            (default runs/full)
    SIMWILD_PARALLEL      models to run concurrently  (default 8)
    SIMWILD_THREADS       threads per model           (default 8)
    SIMWILD_JOB_TIMEOUT   per-model seconds           (default 10800 = 3h)
    SIMWILD_MEM_GB        per-model memory cap, GB    (default 128, 0 disables)
    SIMWILD_LIMIT         process at most N new models (default 0 = all)
    SIMWILD_SAMPLE        name | smallest | spread | random  (default smallest)
    SIMWILD_SEED          seed for SIMWILD_SAMPLE=random  (default 0)
    SIMWILD_REPORT_ONLY   if set, only regenerate the report and exit

Memory note: the cap is PER MODEL, not a budget for the sweep. 8 x 128G is more than
kirby has, so the cap is a runaway-killer, not an admission control -- it stops one
pathological mesh from swapping the box, and does nothing in the normal case. Lower
SIMWILD_MEM_GB if the sweep has to coexist with someone else's job.

Function Documentation

_describe_exit()

run_simwild_sweep._describe_exit (   code,   work  )

protected

Human-readable reason for a nonzero exit, distinguishing the OOM kill.

A cgroup OOM kill arrives as SIGKILL, which is also what a manual kill looks
like -- but we only kill on stop or timeout, and both are handled before this
is reached, so an unexplained SIGKILL under a cap is the OOM killer. systemd
says so on the scope's stderr when it can, so prefer that when it is there.

_histogram()

run_simwild_sweep._histogram (   title,   values,   edges,   fmt = "{:g}"  )

protected

ASCII histogram. edges is a list of bucket boundaries; last bucket is open.

_memory_capped_available()

run_simwild_sweep._memory_capped_available ( )

protected

Probe once, so a broken systemd user manager fails loudly at startup.

_order_models()

run_simwild_sweep._order_models (   meshes )

protected

Order the work list. Only matters when the run is truncated (LIMIT / a stop).

smallest: cheapest first, so a partial run covers as many models as possible and
          the timeout mostly bites the giant meshes, which come last. This is the
          right order for the full sweep.
spread:   walk the size-sorted list at a stride, so a truncated run samples the
          whole size range. A 100-model trial ordered 'smallest' tells you almost
          nothing -- the 100 smallest meshes in Thingi10K are trivial.
random:   uniform sample, seeded by SIMWILD_SEED.
name:     filename order. Size is uncorrelated with the name, so the expensive
          models are spread through the run instead of all landing at the end --
          which keeps the machine busy to the finish and makes progress linear
          rather than front-loaded. This is what the 2D runner defaults to.

_parse_success()

run_simwild_sweep._parse_success (   d )

protected

time, iterations, energies for one successful model.

generate_html_report()

run_simwild_sweep.generate_html_report	(		rows,
			stats,
			report_dir,
			out_dir,
			dataset_dir = None,
			mesh_extensions = ()
	)

Write report/index.html. `rows` are the same dicts the CSV is built from.

run_one()

run_simwild_sweep.run_one

(

mesh_path

)

Run simwild on one mesh. Returns 'success' / 'failure' / 'stopped'.

stop_sweep()

run_simwild_sweep.stop_sweep

(

)

Signal a running sweep to stop cleanly (via its pidfile).

Variable Documentation

_JS

str run_simwild_sweep._JS

protected

Initial value:

=  """
(function () {
  var tip = document.getElementById('tip');
  document.querySelectorAll('[data-tip]').forEach(function (el) {
    el.addEventListener('mousemove', function (e) {
      tip.textContent = el.getAttribute('data-tip');
      tip.style.opacity = '1';
      var x = e.clientX + 14, y = e.clientY + 16;
      var w = tip.offsetWidth, h = tip.offsetHeight;
      if (x + w > window.innerWidth - 8) x = e.clientX - w - 14;
      if (y + h > window.innerHeight - 8) y = e.clientY - h - 16;
      tip.style.left = x + 'px'; tip.style.top = y + 'px';
    });
    el.addEventListener('mouseleave', function () { tip.style.opacity = '0'; });
  });
})();
"""

PARAMS

dict run_simwild_sweep.PARAMS

Initial value:

=  {
    "application": "simwild",
    "eps_rel": 2e-3,
    "num_threads": THREADS,
    # Deliberately *not* the spec default (which is true). Requested for this sweep:
    # with preserve_topology off, simplification runs pre-insertion and is allowed to
    # change input topology, which is the configuration under test here.
    "preserve_topology": False,
    # The .msh is the real output and the .vtu is a visualization dump that _prune
    # deletes immediately afterwards, so writing it costs time and disk on every model
    # for nothing. The 2D sweep has always forced this off.
    "write_vtu": False,
}