build_a_blog: bench, main errors

This commit is contained in:
Collin Lefeber 2024-06-19 15:34:10 -04:00
parent d2b810441d
commit b44ec21007
3 changed files with 20 additions and 12 deletions

View file

@ -16,6 +16,11 @@ echo "INFO: number of *.md files $(find ./posts/ -iname '*.md' | wc -l)" 1>&2
echo "INFO: number of *.html files $(find ./posts/ -iname '*.html' | wc -l)" 1>&2
echo "INFO: running" 1>&2
time -p python main.py 2>/dev/null
rc=$?
if [[ "$rc" != "0" ]]; then
echo "ERROR: program exited with $rc" 1>&2
exit 1
fi
echo "INFO: number of *.html files $(find ./posts/ -iname '*.html' | wc -l)" 1>&2
echo "INFO: cleanup __bench files" 1>&2
rm -f ./posts/*__bench*

View file

@ -59,7 +59,7 @@ def render_posts():
with Pool(processes=cpu_count) as pool:
posts = pool.map(render_post, files)
logging.info("render_posts result: %s", res)
logging.info("render_posts result: %s", posts)
return posts
def posts_list_html(posts):

View file

@ -757,6 +757,11 @@ echo "INFO: number of *.md files $(find ./posts/ -iname '*.md' | wc -l)" 1>&2
echo "INFO: number of *.html files $(find ./posts/ -iname '*.html' | wc -l)" 1>&2
echo "INFO: running" 1>&2
time -p python main.py 2>/dev/null
rc=$?
if [[ "$rc" != "0" ]]; then
echo "ERROR: program exited with $rc" 1>&2
exit 1
fi
echo "INFO: number of *.html files $(find ./posts/ -iname '*.html' | wc -l)" 1>&2
echo "INFO: cleanup __bench files" 1>&2
rm -f ./posts/*__bench*
@ -807,7 +812,7 @@ So here is the diff to make that happen:
```diff
diff --git a/main.py b/main.py
index 52ce57b..91d852b 100644
index 52ce57b..95b650d 100644
--- a/main.py
+++ b/main.py
@@ -1,9 +1,11 @@
@ -842,7 +847,7 @@ index 52ce57b..91d852b 100644
+ with Pool(processes=cpu_count) as pool:
+ posts = pool.map(render_post, files)
+ logging.info("render_posts result: %s", res)
+ logging.info("render_posts result: %s", posts)
return posts
def posts_list_html(posts):
@ -857,9 +862,9 @@ INFO: removing old __bench files
INFO: number of *.md files 102
INFO: number of *.html files 2
INFO: running
real 0.21
user 1.64
sys 0.14
real 0.27
user 1.82
sys 0.15
INFO: number of *.html files 102
INFO: cleanup __bench files
@ -868,16 +873,14 @@ INFO: removing old __bench files
INFO: number of *.md files 1002
INFO: number of *.html files 2
INFO: running
real 1.12
user 15.69
sys 0.24
real 1.25
user 16.68
sys 0.52
INFO: number of *.html files 1002
INFO: cleanup __bench files
```
So down to <1s for 1000 posts with 16 cores (`multiprocessing.Pool` uses cpu core count by default if not passed) .
🎉 that's cool!
Down to 1.25s for 1000 posts 🎉
[1]: https://crystal-lang.org/
[2]: https://github.com/crystal-lang/crystal/releases/tag/0.31.0