In the past HFSS on Flux and Nyx systems exhibited strange parallel performance behavior. This was because of a bug in an old library in HFSS interacting with a feature we use on the clusters.
After about two years of communication the new hfss has been validated with a newer version of this library and appears to work correctly. Instructions follow how to use hfss 2014.0.0 with Flux and Nyx to get the expected performance.
Use the newer hfss:
module load hfss/2014.0.0ANSOFT_NUM_PROCESSORS and ANSOFT_DESIRED_RAM_LIMIT nolonger do any thing in this version.
To set the number of cores to use, which is the same value as N in nodes=1:ppn=N use:
#update NumCoresPerDistributedTask=N
hfss -ng -batchsolve -batchoptions "HFSS/NumCoresPerDistributedTask=N HFSS/UseLegacyMultiprocessingLicense=1" Test1.hfss
This should correctly use all the CPUs.
Also because of an updated math library on newer nodes you might see for the same core count close to 2x performance improvement for those routines only. So you should expect less than that but you might get some speedup from things other than cpu count.
Example PBS script for hfss/2014.0.0
UPDATE: We now support Distributed Memory (DDM, DSO) HFSS
