The English book printed project: production report 1

Published on


After the release of the eShop (thank you for all the purchases!). I started this week another huge item on my TO-DO list for Pepper&Carrot: self-publishing it as a comic book. I do it because I find it exciting to make a big high quality comic book using only FLOSS from scratch! I'm not expecting much from the sales. I just want my own Pepper&Carrot book, 100% FLOSS to crown the last five years of production. Focusing on that target even drives me away from the Inktober challenge I usually very like. But it's a project with a lot of difficulties and I want to make it perfect. So I took the necessary time to do research and experiments before going into production. And that's where I am at start of October 2019. Here is the results of one week of intense research and tests:

The book

Deciding the format was hard: I bought samples done by my printer, reviewed all book at home. After a lot of thought on it, my project will be a large 8,5"x11" (21,59x27,94cm) and thick book with a hard-cover. It will include 193 pages of comics -from episode 1 to episode 29- printed in full resolution on a high-quality almost mate paper inside and glossy for the cover. ( I have a sample of this quality on my desk, this is really what I want). A gallery of additional artworks will be added on full page to the comic, full credits and the list of patrons to shape a 200 pages large book. My target dream price: under 30$ (without shipping). I'll see if I can do that, if I can't I'll explain why in a future update. So far, it looks like I can make it from my estimation.



A double page in Scribus

A dedicated cover

I want the book to have its own cover, so I plan to paint a specific illustration for it. It will take me days but I don't want to reuse the illustrations already in usage for the Glénat book project, the TokyoPop publishing, the Breton version, etc... I'll start this artwork next week and I'll show the progress on social medias.

100% made with FLOSS

"[...] there's an entire industry out there that spent the last ~25 years making sure it's a pain in the rear to do anything useful with PDFs without their expensive tools. Now you want to undermine their collective efforts? Think of the children! :)"
~ Satō Katsura (from here).
This quote summarize perfectly the situation. As usual, I'll use only Free/Libre and open-Source software: Krita, Inkscape, Scribus on my Kubuntu 18.04.2 LTS. Yet, my printer provides a tutorial for Scribus, the *.icc and templates. (A friendly-FLOSS printer? Only when I'll receive perfect prints I'll be able to tell).


Testing various CMYK convertion methods (comparing a CMYK red from Krita and CmykTools)

Multilingual, collaborative on Git

English only? What about Spanish? Russian? or more? That's one of the most ambitious part of my project: I started to design a multilingual system for the maximum automation around a GIT repo with scripting. I plan to use the data already translated in order to publish a lot of languages; but I'll also need help to translate specific texts. This will come after, I'll start with the English version


An early brainstorming/mind-map of the translation system for the book publishing repository.

Printer specifications

My printer works only with the CGATS21_CRPC1.icc CMYK color profile and the PDF/X-1a or PDF/X-3 file format. I can't expect in this process a man in the middle reviewing my file on a machine with proprietary software like the one of Adobe: everything goes direct to the print machine: it must be compliant and perfect. If anything fails, I'll have to pay for the mistake and the time of the operator. Also, PDF/X-1a or PDF/X-3  mean no transparency  for designing the page.




Without transparency, I'll have to overlay page with translation and without to clean credits.
(Example gif animation of the process)

The test and bugs:

During the process of exporting dozens of test PDFs with various settings and trying everything the web wrote on the topic with trial and error, I made discoveries. I report them here if you are curious enough to read them. But also for them to get referenced later by search engine; I'm also posting my workarounds: they might help.



Rendering comparison (from left to right):
Scribus (Softproofed with CGATS21_CRPC6.icc but output with
CGATS21_CRPC1.icc)
Krita displaying the extracted image from PDF
Okular displaying the PDF output (not color managed)
on right, Nomacs displaying the original sRGB.

The fog of CGATS_CPR1.icc

Haaaa, color management on Linux... All a poetry! The color space of the profile looks really small at a first glance, but according to the sample I bought; it is very capable of wonderful printed comics colors. Unfortunately, the Soft Proofing of this CMYK profile renders an horrible bleached rendering, feeling like being lost in the fog of a haunted wood. This rendering is consistent in Gimp, Krita and Scribus and it took me hours of test to exclude I was doing a mistake with my color workflow.

But once rendered the issue disappear, and colors looks good as a CMYK image can be (so after a real convert, not a soft-proof preview). This is also consistent in Krita and Scribus (not in Gimp because well.. still no CMYK as far as I know).

So, no possible direct WYSIWYG workflow with Soft Proofing (What You See Is What You Get) but a WYSHITFOG workflow (What You See Happens In The Fog Of Gamma).  I joke. I found a workaround: Switch to another CMYK profile during prod ; after hours of research I found that the CGATS_CPR6.icc while being softproofed looks similar CGATS_CPR1.icc rendered (not 1:1, but visually Ok). Note for later: don't forget to switch back for export.

Bug Report URL (Krita):
[CMYK] CGATS21_CRPC1.icc has a different rendering between softproofed and converted :  https://bugs.kde.org/show_bug.cgi?id=412604


Soft Proofed preview VS real convertion with CGATS21_CRPC1.icc.

The 72ppi TIFF of Scribus

When I started to get a somehow functional color managed workflow, It was obvious I'll have to bring modifications to the comic pages and adress issues I could spot with the CMYK profile (too dark pages, out of gamut effect that translate badly to CMYK,etc). That's something no  other publisher did with Pepper&Carrot (certainly to 'respect the art') and they often just went at printing the RGB as it is with more or less success (and sometime, disasters for certain crowdfunding). Well, I'll want to adapt the art and I can.

So, I ran a series of test to evaluate how I could save my modifications directly in CMYK and get them in Scribus. I decided to pre-convert pages to CMYK in Krita and export them as Tiff then import them on Scribus: this was fixing the "preview in the fog" workflow but I discovered Scribus was skipping the resolution ppi of Tiff and import them as 72ppi by default. I then tried to update my version. I tested also the last version in development. All of them had this issue, this is too hard to work like that for 200 pages, too many manual tweaks and I wasn't feeling confident for the TIFF format in general. So better to drop this option.

Bug Report URL (Scribus) :
0015837: Import of Tiff files: missing resolution/dpi/ppi
[quick edit : It turns out Krita is guilty to not write the unit "inch or centimeters" while writing the Tiff]


Comparing the same page with various convertion method in Scribus: original on right.

Remastering the artworks on pages for print


On newer tests, it has appeared the best cross-format for CMYK pictures between Krita and Scribus was 'CMYK JPGs'. That's really curious; this format was a taboo for my teachers when I learned Pre-Press with QuarkXPress and Indesign around 2000 (in my youth). But it works and with a compression at lossless (100% quality) it is even a good solution for a transition flat format. Things I learned. But I don't feel good about using this format: thumbnails on the file browser looks terrible and it still feels like a hack in a way.

Anyway, I discovered on the process that -with the right settings- the export to PDF of Scribus was doing the conversion to CMYK on the fly whatever input was entered on Scribus; including sRGB format. Splendid technology. I then tested the quality of the exported PDF CMYK of Scribus in competition with Krita and Cmyktools. I obtained very similar results with Scribus and Krita. Cmyktools had other very interesting optimization, but the software being not maintained since 2011, the GUI was spanning accross my two monitors and the resulting CMYK was color inverted after extraction from the PDF. I had to exclude it. The quality of auto-converted CMYK via Scribus had similar (if not exact) quality of the one converted by Krita. I guess it is the same LittleCMS magic under the hood for both of them.

I then decided to go with only the 8Bit RGB workspace for my modification: unify my artworks for web and for print in a single format and remaster the sRGB with Soft Proofing activated for CGATS_CPR1.icc (in fact, 'workarounded' with CGATS_CPR6.icc to avoid the fog).  Tweaking slightly the RGB will mean modifying Pepper&Carrot online pages too or how the episodes were designed at release. Remastering them for the book project is the right effort to do at the source to avoid CMYK mistake for the future converting while keeping always the sources artwork in a single place.


Left: Original, Right: the type of RGB corrections that might happens.
(work in progress, not happy with the contrast yet).

Validation of the PDF for print

On testing the output PDF of Scribus, the harder is to know when I produce a valid PDF export. How to inspect it? I have not found a perfect solution... I still blindly have to trust on the output of Scribus. Knowing the buggy nature of FLOSS and how the project degenerate quickly this is not making me confident.  But I found a couple of options, mostly via command line interface, on a GNU/Linux system:

Imagemagick:

Imagemagick can list a lot of information about the PDF with the command under, including the CMYK nature of the PDF. Unfortunately, no color space. Does it mean ImageMagick doesn't do it? Or Scribus skip including it? I'll never know.

$ identify -verbose output.pdf

Imagemagick can also convert a page of the PDF to a PSD. This command under will return a wrong resolution PSD (a thumbnail) and Krita can open it. CMYK values are well set on the PSD, but the color profile is lost: Krita fallback to Chemical Proof profile: showing something quite similar to Poppler.

$ convert output.pdf output.psd

Poppler:

Poppler via Poppler-utils can list all the image inside a PDF and report if they are CMYK. Same, no color profile, so it can be CMYK-pizza or CMYK-chocolate, this is not Poppler-utils business. I would really like to know if they are all profiled with CGATS_CPR1.icc or if the PDF as a global file is tagged with CGATS_CPR1.icc...

$ pdfimages -list output.pdf

A useful output!

It is also possible to extract the content of the PDF to a folder using the Poppler-utils this way:

$ mkdir extracted
$ pdfimages -all output.pdf extracted/img
The Tiff, PPM or JPG obtained this way are CMYK files but unfortunately have no profile so Krita will display them with the default generic free/libre CMYK color profile: ChemicalProof. You'll need to convert them manually to CGATS_CPR1.icc with Imagemagick (apply simply the profile; no convert/scaling):

$ cd extracted
$ convert img-000.tif -profile /path/to/your/CGATS_CPR1.icc img-profiled.tif

Then you can open in Krita and control the quality of the Scribus rendered files with the color picker to see how it manages the black, how out of gamut colors are scaled, etc.... That's my best solution so far!

Okular:

Okular, the Plasma desktop PDF reader can read the PDF thanks to Poppler under the hood. But same error here: the CMYK colors are not profiled and fallback to a default profile like Chemical Proof. It is still a blast to have the possibility to read a version of a CMYK PDF on a PDF reader, even if the colors are not faithful: the picture are still readable.

Krita:

Krita can open a page of the PDF;  but will do a bit the same than a Poppler conversion: the picture has same generic CMYK rendering and the result is even not CMYK anymore but all converted in the default RGB used by Krita (sRGB-elle-V2-srgbtrc.icc)...

Conclusion and more links:

I know a lot of you will skip reading this long log and that's fine. I wrote this long results of my research for a future person who will get same trouble as I do and will search Internet to see if other tempted this quest. That's a bottle in a ocean, and a way to log my efforts on the way. 

Here are my sources and best Internet links I could find on the topic. I triaged them from reading hundreds of pages.

COLOR MANAGEMENT:

Krita documentation: Soft Proofing
https://docs.krita.org/en/user_manual/soft_proofing.html

Krita documentation: all sub-chapters of "Colors" category, a must read.
https://docs.krita.org/en/general_concepts/colors.html

Elle Stone website: Nine Degrees Below, a goldmine for every articles.
https://ninedegreesbelow.com/

FLOSS Manual : Scribus. One of the easiest guide into Scribus.
http://write.flossmanuals.net/scribus/introduction/

Scribus Wiki: Color Management setup, especially good to read after reading Krita documentation on the same topic.
https://wiki.scribus.net/canvas/Color_Management_setup

Fedora Project wiki: How to set CMYK color on a design for printing. A practical guide, a bit outdated but still nice because practical.
https://fedoraproject.org/wiki/How_to_set_CMYK_color_on_a_design_for_printing

Preparing Your Book For Print with Scribus goldmine documentation from my printer:
https://onebookshelfpublisherservice.zendesk.com/hc/en-us/articles/360022742353-Preparing-Your-Book-For-Print-with-Scribus

PDF:

The PDF/X-1a file format, to know more about the compliance.

Unix.stackexchange.com : What are GNU/Linux tools for checking PDF documents before publishing? To know more about the tools available.
https://unix.stackexchange.com/questions/316760/what-are-gnu-linux-tools-for-checking-pdf-documents-before-publishing

Adobe Support Community - Transparency Compliance with PDF/x-1a ; to understand why no transparency in this format.
https://community.adobe.com/t5/InDesign/Transparency-Compliance-with-PDF-x-1a/td-p/9277582

Thatch Tran: Improving PDF export in Scribus, to understand a bit more the situation.
https://wiki.scribus.net/canvas/Thatch_Tran:_Improving_PDF_export_in_Scribus

DOWNLOADS:

INTERNATIONAL COLOR CONSORTIUM, CGATS21-2-CRPC1:

INTERNATIONAL COLOR CONSORTIUM, CGATS21-2-CRPC6:

The repository of CMYKTools by BlackFiveImaging
https://github.com/blackfiveimaging/cmyktool


License: CC BY
David Revoy, www.davidrevoy.com, .
Unless otherwise mentioned in the article.

Tags:  #lab  

25 comments

link Anonymouse   - Reply

While I'm not versed in graphical design, I do so enjoy seeing the progress you are making getting an English version of Pepper & Carrot to comic readers. I want it! =3

But oh no! No Inktober! Sadness!!

link David REVOY Author, - Reply

Thank you! Hehe, I'll try to find time to do one or two inked drawing maybe ;)

link Francisco   - Reply

Very interesting everything you expose in this article, although I think I will need to reread it several times, since it is dense and contains a lot of information. Currently, I work on an illustrated album, which is actually a somewhat stuck project for some years, and your work is a reference for my project. Congratulations!

I have observed that you use the Nomacs image viewer, instead of the default KDE Plasma, Gwenview. Currently, my OS is Manjaro KDE, and Gwenview is very useful when previewing .kra files, and can even manage my monitor profile correctly. However, it seems to ignore the periles embedded in Krita sRGB files. Is Nomac able to do full color management?

link David REVOY Author, - Reply

Thank you very much. Yes, I'm using Nomacs; the reason over Gwenview is the compact GUI and the navigation (middle mouse to zoom/pan). For the issue you had about color management with Gwenview and *.kra this is not the fault of Gwenview; Nomacs has the same behavior (they use the same libs underneath). The Krita preview is just a mergedimage.png at the root of the *.kra file (a zip). This PNG is like exactly a RGB 8Bit version of how look the projection of the file on Krita viewport at the moment of saving. It's a projection as pure RGB 8bit, not really the real datas (that can be CMYK/L*a*b/etc...)

link Leo Plaw   - Reply

Thank you for putting in all of the hard work to figure this out David. FOSS needs people like you to thoroughly put it through it's paces to figure out the short comings so a profession workflow can be built.

link David REVOY Author, - Reply

Thank you. Yes, all this paper-cuts in my article feels so minors compare to what is done so far and the possibilities of this tools. Unfortunately, I know that this is the typical kind of paper-cuts that would rool the eyes of a professional while testing it for the first time. I'm glad I can find a way with workaround but it took a lot of time. I will be happy if it happens one day for my workarounds to be replaced by real features. FLOSS tools are very capable. But a ton of comfort is missing and many bugs are hard to accept in a professional environment. Maybe prepress is an industry where the userbase think the things have to work as in a factory?

link Pierre Schiller   - Reply

I've read it all. Thank you for making it such detailed text. I too come from a print background, so I can relate to all you have to exposed here. Addessing CMYK on pdf generated by open source, I guess it kind of is the "authorship" and technology licencing from 30 years ago. Fortunatelly the audio visual entertainment industry through The Linux Foundation had gotten far as to use open source solutions and formats and profiles. Let's just hope that becomes a reality in printing.

link David REVOY Author, - Reply

Thank you for the comment. I hope too!

link zeograd   - Reply

It's great to see you document the FLOSS way to do professional comic publication.
Do you know which part exactly could the community focus on to improve the process ?

link David REVOY Author, - Reply

Hey Zeograd! Thank you.
No idea on what could improve the process. I guess I would feel better if I had the alternative to "Adobe Acrobat Pro"; from what I read on the usual prepress workflow, one does the desktop-publishing work with InDesign, output as PDF then can load the PDF into "Adobe Acrobat Pro" and check if the PDF is perfect (no missing image, good layout, good color profile, compliant with a PDF standard via a box that list all the metadata/tags of the PDF). On FLOSS; Okular (using Poppler in background) is not so far to that. But the "color management" is missing. And only for that bit, I can't tell if color management is missing in Okular, or if Scribus malformed my PDF at output. Without any tool to diagnosis a PDF, I think it will be hard to do serious work. For sure, I'm taking a risk when I'll have to export the 200 pages with Scribus that will weight probably 1GB without being able to see how look my PDF. I'll have to blindly trust in getting a good, perfect, rendering.

link Great job!   - Reply

You are tracing a new route: hope you reach success and happyness.

link TappedOut   - Reply

Is the (missing) color profile the only issue, you know of to work out?

link David REVOY Author, - Reply

That's the main issue :) No idea if Scribus will send a PDF without the good color profile embeded or if it is there. I have no tools to tell; and I don't want to buy a copy of Windows with Acrobat Pro just to check. I'll probably try to read the file in a Hex editor and look for the text string.

link TappedOut   - Reply

Can you crowd-source that to people that do, or does that go against the spirit of your project?

link David REVOY Author, - Reply

Good idea, I'll add links to my repositories and suggestion about how to help on next reports.
For sure, I'll open the PDF to the community and source *.sla will get its repository; proofreading the book by fresh eyes is an essential step in this project.

link Châu   - Reply

Also try hexdump command line tool from terminal

link Vulphere   - Reply

Some interesting technical information there, especially regarding the colour profile and challenges to ensure accurate colour reproduction.

link David REVOY Author, - Reply

Thank you!

link saski'o   - Reply

I'm excited to hear more about the process for getting prints in other languages once you figure that out, and what the automation will be like.
I'd like to get a copy of the book in zbalermorna once I finish helping out with that translation.

link David REVOY Author, - Reply

For sure I'll share all!
Yes, Lojban is on my top list and I spoke with Gleki about it.
I don't know if it means the one with Lojban font, or with the latin characters. But it will be an interesting quest; and a premiere!

link saski'o   - Reply

It's the special font. I've been talking with the guy maintaining the font and it looks like it's been moved to a new unicode space. But yes, it'll be very interesting.

link Doudou   - Reply

Wow, ça c'est de la contribution tous azimuts au libre ! Bravo !

link David REVOY Author, - Reply

Merci :)

link Samuel A Rowan   - Reply

I recently discovered your work through Xacur's Kickstarter campaign and hearing that these lovely comics will be printed excites me greatly! I'll be sure to check back often to see when it drops!

link Châu   - Reply

PDF is Adobe's dog. I wish can create more modern system use SVG replace PDF. Another good bitmap image format for replace TIFF (another Adobe dog) is OpenEXR but I never see any person use it for print (or CYMK file). It support 16 bit and 32 bit float color for wide dynamic range. OpenEXR format support any channel name.

Probably your printer people also have other software reorganize your book pages, order for one big page have 8 or 16 book page (or other number) for print at one time, then they fold it for cut become book. This software is probably close source. Also final raster software for their printer. Can build full open source system if can get open source printer. Diffusion and screen algorithms is available in graphics research papers.

Leave a reply

Comments will be moderated according the Citizens of Hereva Code of Conduct.

CapchaCapcha

Enter image code