Guide How to: reduce HTML weight/length from the core/java

Salty Mike · November 7, 2023

Greetings, folks!

Here is a way to optimise the HTML length by up to 30%, depending on the nesting.

The examples I give are taken from an old Interlude core, but the general idea is applicable to all java-based cores.

Proof of concept!
The two important bits in the screenshot below are the (1) Original HTML size row and the (4) No Tabs/Rows size row, which represent the before and after, respectively.

STEPS:

1. locate the `setHtml(String text)` method inside `NpcHtmlMessage.java`.

2. add the following bit of code as a standalone method:

/**
 * Replaces all occurrences of New Rows and Tab spaces in the string.
 * @param text the string that may contain invalid characters
 * @return the string with invalid character replaced by underscores
 */
public static String removeTabSpacesAndNewRows(String text)
{
	return text.replaceAll("[\\t\\n]", "");
}

3. find the last ocurence of the `text` String variable in the `setHtml()` method, and push it through this newly created method like so: `removeTabSpacesAndNewRows(text);`.

4. example of the final result.

/**
 * Sets the html.
 * @param text the new html
 */
public void setHtml(String text)
{
	if (text == null)
	{
		LOGGER.warning("Html is null! this will crash the client!");
		_html = "<html><body></body></html>";
		return;
	}
	
	if (text.length() > 8192)
	{
		LOGGER.warning("Html is too long! this will crash the client!");
		_html = "<html><body>Html was too long,<br>Try to use DB for this action</body></html>";
		return;
	}
		
	_html = removeTabSpacesAndNewRows(text); // html code must not exceed 8192 bytes
}

/**
 * Replaces all occurrences of New Rows and Tab spaces in the string.
 * @param text the string that may contain invalid characters
 * @return the string with invalid character replaced by underscores
 */
public static String removeTabSpacesAndNewRows(String text)
{
	return text.replaceAll("[\\t\\n]", "");
}

5. You could further refine it by processing the `text` variable between the two IF clauses by creating a new local variable, assigning it a value of `text` and then replacing the subsequent `text` mentions in the `setHtml()` method with this new local var. Here's what it would look like:

/**
 * Sets the html.
 * @param text the new html
 */
public void setHtml(String text)
{
	if (text == null)
	{
		LOGGER.warning("Html is null! this will crash the client!");
		_html = "<html><body></body></html>";
		return;
	}

	String refinedText = removeTabSpacesAndNewRows(text);
	if (refinedText.length() > 8192)
	{
		LOGGER.warning("Html is too long! this will crash the client!");
		_html = "<html><body>Html was too long,<br>Try to use DB for this action</body></html>";
		return;
	}
		
	_html = refinedText; // html code must not exceed 8192 bytes
}

UPDATE:
(thanks to @xdem for pointing it out)

You can instead apply the same logic/method to the HTMCache.java.
To do so, locate the loadFile() method and recycle the String/Text/Content through the removeTabSpacesAndNewRows method.

KEEP IN MIND that you might want to move the removeTabSpacesAndNewRows method in another java class, preferrably in some UTILITY class.
Here is an example:

	public String loadFile(File file)
	{
		final HtmFilter filter = new HtmFilter();
		String content = null;
		if (file.exists() && filter.accept(file) && !file.isDirectory())
		{
			FileInputStream fis = null;
			BufferedInputStream bis = null;
			try
			{
				fis = new FileInputStream(file);
				bis = new BufferedInputStream(fis);
				final int bytes = bis.available();
				final byte[] raw = new byte[bytes];
				bis.read(raw);
				
				content = new String(raw, StandardCharsets.UTF_8);
				content = content.replaceAll("\r\n", "\n");
				content = content.replaceAll("(?s)<!--.*?-->", ""); // Remove html comments
				content = NpcHtmlMessage.removeTabSpacesAndNewRows(content);
				
				final String relpath = Util.getRelativePath(Config.DATAPACK_ROOT, file);
				final int hashcode = relpath.hashCode();
				if (Config.CHECK_HTML_ENCODING && !StandardCharsets.US_ASCII.newEncoder().canEncode(content))
				{
					LOGGER.warning("HTML encoding check: File " + relpath + " contains non ASCII content.");
				}
				
				final String oldContent = _cache.get(hashcode);
				if (oldContent == null)
				{
					_bytesBuffLen += bytes;
					_loadedFiles++;
				}
				else
				{
					_bytesBuffLen = (_bytesBuffLen - oldContent.length()) + bytes;
				}
				
				_cache.put(hashcode, content);
			}
			catch (Exception e)
			{
				LOGGER.warning("Problem with htm file " + e);
			}
			finally
			{
				if (bis != null)
				{
					try
					{
						bis.close();
					}
					catch (Exception e1)
					{
						LOGGER.warning("Problem with HtmCache: " + e1.getMessage());
					}
				}
				
				if (fis != null)
				{
					try
					{
						fis.close();
					}
					catch (Exception e1)
					{
						LOGGER.warning("Problem with HtmCache: " + e1.getMessage());
					}
				}
			}
		}
		
		return content;
	}

Edited November 14, 2023 by Bru7aLMike
headline

xdem · November 7, 2023

okay its simple and it will work, but you better save the minified html directly in the HTML Cache during the load phase!

Salty Mike · November 7, 2023

9 minutes ago, xdem said:

okay its simple and it will work, but you better save the minified html directly in the HTML Cache during the load phase!

Ok, I will update the guide then.

xdem · November 7, 2023

I would also advice you to experiment with

replaceAll("\\s+", " ");

s+ will automatically match any whitespace character (including \r \n etc)

Salty Mike · November 7, 2023

25 minutes ago, xdem said:
I would also advice you to experiment with
replaceAll("\\s+", " ");
s+ will automatically match any whitespace character (including \r \n etc)

I have, as evident from the posted screenshot, see a copy below.

The bottom-most line is No White-Spaces, at all.
However, the 5% gain is not worth the risk, in my personal opinion. The Tab-Spaces and New Rows reduces the overall length by quite a lot as is.

Here is a list of all four methods that I used in the screenshot.

	public static String removeTabSpaces(String text)
	{
		return text.replaceAll("\\t", "");
	}

	public static String removeNewRows(String text)
	{
		return text.replaceAll("\\n", "");
	}

	public static String removeTabSpacesAndNewRows(String text)
	{
		return text.replaceAll("[\\t\\n]", "");
	}

	public static String removeAllWhiteSpacing(String text)
	{
		return text.replaceAll("\\s+", "");
	}

protoftw · November 8, 2023

That's so useful in interlude clients since the html length is very low and you can't have a full menu page.

For example, this is in interlude but html code after editing to remove spaces and etc is unreadable.
It will make editing way much easier.

Thank you for your share

Projack · December 11, 2023

Depending on client you can additionaly remove useless attributes from html. For examle: border=0

Guide How to: reduce HTML weight/length from the core/java

Recommended Posts

Salty Mike

xdem

Salty Mike

xdem

Salty Mike

protoftw

Projack

Join the conversation

Posts

Topics

Browse

Activity

Store