The Mysterious Intermittent MySQL Crash on a Joomla Website

Most people love mystery movies, there is something thrilling about them. We also love mystery movies, but there is something that we love even more, and it is Joomla mysteries, and yesterday we were fortunate enough that we were assigned to investigate one of these mysteries…

At around eleven hundred hours yesterday morning, a regular client of ours told us that they were seeing the following error on their Joomla site:

Error displaying the error page: Application Instantiation Error: Could not connect to MySQL.

Of course, the above error is probably the most common fatal error on Joomla sites, and we have written about it before, yet this time, it was different, because a few minutes later, the client called back and told us that the site was now working. Hmmm… It was the first time where we see this issue resolve itself. That was weird, we thought. We asked the client if they wanted to investigate further, but they told us that they were OK; it was just a hiccup…

A couple of hours later, they emailed us, and told us that they were seeing the same error message that they saw in the morning. By the time we received the email and checked the site again, the error was gone. So, we emailed the client back, and we recommended that they authorize an investigation about this issue, because instability issues such as these are never a good sign. The client immediately authorized the investigation, and the first thing that we checked was the MySQL error log which is located under the /var/lib/mysql folder (the file name typically ends with .err). Here’s what we saw:

2017-11-07 11:03:29 82870 [Note] Plugin 'FEDERATED' is disabled.
2017-11-07 11:03:29 82870 [Note] InnoDB: Using atomics to ref count buffer pool pages
2017-11-07 11:03:29 82870 [Note] InnoDB: The InnoDB memory heap is disabled
2017-11-07 11:03:29 82870 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2017-11-07 11:03:29 82870 [Note] InnoDB: Memory barrier is not used
2017-11-07 11:03:29 82870 [Note] InnoDB: Compressed tables use zlib 1.2.3
2017-11-07 11:03:29 82870 [Note] InnoDB: Using Linux native AIO
2017-11-07 11:03:29 82870 [Note] InnoDB: Using CPU crc32 instructions
2017-11-07 11:03:29 82870 [Note] InnoDB: Initializing buffer pool, size = 64.0M
2017-11-07 11:03:29 82870 [Note] InnoDB: Completed initialization of buffer pool
2017-11-07 11:03:29 82870 [Note] InnoDB: Highest supported file format is Barracuda.
2017-11-07 11:03:29 82870 [Note] InnoDB: Log scan progressed past the checkpoint lsn 65178299172
2017-11-07 11:03:29 82870 [Note] InnoDB: Database was not shutdown normally!
2017-11-07 11:03:29 82870 [Note] InnoDB: Starting crash recovery.
2017-11-07 11:03:29 82870 [Note] InnoDB: Reading tablespace information from the .ibd files...
2017-11-07 11:03:29 82870 [Note] InnoDB: Restoring possible half-written data pages
2017-11-07 11:03:29 82870 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number 65178300463
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 1 row operations to undo
InnoDB: Trx id counter is 379714816
2017-11-07 11:03:29 82870 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: Starting in background the rollback of uncommitted transactions
2017-11-07 11:03:30 82870 [Note] InnoDB: 128 rollback segment(s) are active.
2017-11-07 11:03:30 7f6cf1e16700  InnoDB: Rolling back trx with id 379714508, 1 rows to undo
2017-11-07 11:03:30 82870 [Note] InnoDB: Waiting for purge to start
2017-11-07 11:03:30 82870 [Note] InnoDB: Rollback of trx with id 379714508 completed
2017-11-07 11:03:30 7f6cf1e16700  InnoDB: Rollback of non-prepared transactions completed
2017-11-07 11:03:30 82870 [Note] InnoDB: 5.6.38 started; log sequence number 65178300463
2017-11-07 11:03:30 82870 [Note] Server hostname (bind-address): '*'; port: 3306
2017-11-07 11:03:30 82870 [Note] IPv6 is available.
2017-11-07 11:03:30 82870 [Note]   - '::' resolves to '::';
2017-11-07 11:03:30 82870 [Note] Server socket created on IP: '::'.
2017-11-07 11:03:30 82870 [Note] Event Scheduler: Loaded 0 events
2017-11-07 11:03:30 82870 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.38'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)
2017-11-07 13:11:21 11675 [Note] Plugin 'FEDERATED' is disabled.
2017-11-07 13:11:22 11675 [Note] InnoDB: Using atomics to ref count buffer pool pages
2017-11-07 13:11:22 11675 [Note] InnoDB: The InnoDB memory heap is disabled
2017-11-07 13:11:22 11675 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2017-11-07 13:11:22 11675 [Note] InnoDB: Memory barrier is not used
2017-11-07 13:11:22 11675 [Note] InnoDB: Compressed tables use zlib 1.2.3
2017-11-07 13:11:22 11675 [Note] InnoDB: Using Linux native AIO
2017-11-07 13:11:22 11675 [Note] InnoDB: Using CPU crc32 instructions
2017-11-07 13:11:23 11675 [Note] InnoDB: Initializing buffer pool, size = 64.0M
2017-11-07 13:11:23 11675 [Note] InnoDB: Completed initialization of buffer pool
2017-11-07 13:11:23 11675 [Note] InnoDB: Highest supported file format is Barracuda.
2017-11-07 13:11:23 11675 [Note] InnoDB: Log scan progressed past the checkpoint lsn 65220975371
2017-11-07 13:11:24 11675 [Note] InnoDB: Database was not shutdown normally!
2017-11-07 13:11:24 11675 [Note] InnoDB: Starting crash recovery.
2017-11-07 13:11:24 11675 [Note] InnoDB: Reading tablespace information from the .ibd files...
2017-11-07 13:11:27 11675 [Note] InnoDB: Restoring possible half-written data pages
2017-11-07 13:11:27 11675 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number 65221104018
2017-11-07 13:11:30 11675 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
2017-11-07 13:11:33 11675 [Note] InnoDB: 128 rollback segment(s) are active.
2017-11-07 13:11:33 11675 [Note] InnoDB: Waiting for purge to start
2017-11-07 13:11:33 11675 [Note] InnoDB: 5.6.38 started; log sequence number 65221104018
2017-11-07 13:11:33 11675 [Note] Server hostname (bind-address): '*'; port: 3306
2017-11-07 13:11:34 11675 [Note] IPv6 is available.
2017-11-07 13:11:34 11675 [Note]   - '::' resolves to '::';
2017-11-07 13:11:34 11675 [Note] Server socket created on IP: '::'.
2017-11-07 13:11:35 11675 [Note] Event Scheduler: Loaded 0 events
2017-11-07 13:11:35 11675 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.6.38'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server (GPL)

You can clearly see that within 2 hours, MySQL restarted twice, by itself! But why? This is where this whole thing started to spin into a fun Joomla mystery! We looked at all the logs, for a sign that can lead us to the real cause of the problem, and after many hours of investigation, we found a clue. It was the following lines in the messages file which is located under the /var/log folder:

Nov  7 11:03:21 host kernel: [42008818.160677] Out of memory: Kill process 39364 (mysqld) score 27 or sacrifice child
Nov  7 11:03:21 host kernel: [42008818.161356] Killed process 39364, UID 498, (mysqld) total-vm:1402564kB, anon-rss:16576kB, file-rss:252kB
Nov  7 11:03:22 host kernel: [42008818.858715] php invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0

If you carefully read the above lines, you will understand what was happening. The system ran out of memory, and it had to make a decision: either to kill the mysqld process or to sacrifice its child(ren). In the next line, you can see that the kernel decided to kill the mysqld process and immediately went ahead with this decision, which resulted in restarting the MySQL server.

OK, now we know it’s a memory issue. But why? Doesn’t their server have enough memory? We issued the following command in the shell to know the amount of memory the server has…

free -m

… and it returned the following:

             total       used       free     shared    buffers     cached
Mem:          1815       1545        270          0        130       1004
-/+ buffers/cache:        409       1405
Swap:         1999        159       1840

Aha! So this server has less than 2 GB of RAM (it is obviously a VPS, and not a server), and 0% of this RAM was free. Of course, Linux tends to allocate unused memory for cache to speed things up, but still, 2 GB is obviously too small for this website, as the server is running out memory.

We offered 2 solutions to the client: 1) We can optimize the queries on the site in order to reduce the memory usage on the database server, or 2) They upgrade to a real dedicated server with at least 16 GB of RAM.

The client opted for the latter solution, which is, in our opinion, the ideal solution.

So there you have it, if you have intermittent downtimes on your MySQL server, then check your logs (especially the /var/log/messages file) to see if your server is running out of memory. If that’s the case, then the best route that you can take is to add resources to your server (or migrate to a more powerful server that has more RAM). If you want to take an alternative route, then just contact us. We are always ready to help, our fees are right, and our work is ultra professional!

7 Reasons Your Joomla Site Is Suddenly Displaying a Blank Page

You haven’t done anything on your Joomla website for 4 weeks, and you are the only authorized person that has access to the backend of the website and to the site’s filesystem/database. Yet, you wake up this morning and you find out that your website (frontend and/or backend), is displaying a blank page. You call your host, and they tell you they know nothing about it. You panic, you google this issue, and you land on this post. You did the right thing, because in this post, we’ll reveal the top 7 reasons that may have caused the blank page that you’re now seeing on your Joomla site.

  1. Your website got hacked: In the majority of cases, a sudden blank page on a Joomla website is caused by a hack that didn’t go very well; the hacker tried to inject malicious code on your site, but instead, crashed the website. Look at the bright side of this, if the hack fully succeeded and your website still worked, then it would have taken you a long time to discover that your website was hacked, which may cause Google to penalize your website (it is not fun to get out of a Google penalty, and sometimes, you are stuck there for a very, very long time).
  2. Your environment has changed: The second common reason that can cause a sudden blackout (or a blankout) on your Joomla site is an environment change. For example, a MySQL update and/or a PHP update that is (are) not supported by your current version of Joomla. From our experience, asking the host to revert the update is a futile endeavor (although some hosts do have some tools that will allow you to use a different version of MySQL and/or PHP), so the best solution is to either update your website to the latest version of Joomla, or hire someone to resolve the incompatibilities between your Joomla website and the updated environment. Keep in mind that at times the issue is caused by one ore more 3rd party extension that are incompatible with the new version of MySQL and/or PHP.

  3. Your system/database ran out of space: On every page load, Joomla writes something to the database (typically in the #__session table). If Joomla cannot write to the disk for one reason or the other, then it will fail and will display a blank page (unless you have error reporting turned on, in which case Joomla will display the actual error). The solution to this problem is to delete the temporary/unnecessary files (especially obsolete backups) and reduce the size of the database by removing any backup tables/databases. You should also consider asking your host to increase your disk space or your database limits.

  4. You have some corruption at the database level: As stated in our last point, Joomla writes to the database on every page load. If there is a corruption at the database level or at a table level, then Joomla will display a blank page. There are several causes behind a database/table corruption, including, but not limited to, disk failure and bad sectors (which are mainly caused by power outages and forced shutdowns). InnoDB databases tend to fix themselves on database restart (if the cause of the issue is not a disk failure). MyISAM databases, on the other hand, must be repaired manually (which is another reason why you should switch the storage engine of your Joomla database from MyISAM to InnoDB).

  5. You server has hardware issues: Hardware issues include, but are not limited to: disk failure, motherboard failure, bad RAM, and overheating. Only the data center can fix any hardware issues you may have on your server. If you want to minimize the risk of having hardware issues, then you need to migrate to another, newer server every 12-18 months. Unfortunately, we know that most readers will take this advice with a grain of salt, and that they will only remember reading about it on the itoctopus blog when it’s too late.

  6. One of your Joomla plugins went berserk: One of our clients had a system plugin that loaded some content from a remote site. All of a sudden, the content format on the remote site changed and the plugin went berserk, causing the whole Joomla website to crash. Fixing the problem consisted of disabling the plugin (the client didn’t want it anymore). Admittedly, we only had this one case where a system plugin was the cause of the issue, so the probability for this happening on your site is very low. Still, it is not a bad idea to review your system plugins if the cause of your problem is none of the above.

  7. You have once worked with a wicked developer: It’s a wicked, wicked world and there are some wicked, wicked Joomla developers out there. It might be that a developer added some code somewhere on your website a long time ago that will make it die on a specific date. Fortunately, debugging this issue is not hard if you hire the right people to do it for you (such as us).

We hope that you found this post useful, and that it helped you address the sudden blank page issue on your Joomla website. If, however, your problem is not caused by anything in the above list, or if you need help with the implementation, then please contact us. We are here to help, our fees are always affordable, we are very reliable, and our work is very clean!

A Simple Yet Powerful K2 Optimization Tip for Joomla Sites

While optimizing an already optimized K2 powered Joomla website yesterday, we noticed the following query in the slow query log:

SELECT i.*, c.name as categoryname,c.id as categoryid, c.alias as categoryalias, c.params as categoryparams FROM #__k2_items as i RIGHT JOIN #__k2_categories AS c ON c.id = i.catid WHERE i.published = 1 AND i.access IN(1,1,5) AND i.trash = 0 AND c.published = 1 AND c.access IN(1,1,5) AND c.trash = 0 AND ( i.publish_up = '0000-00-00 00:00:00' OR i.publish_up <= '2017-10-31 14:53:20' ) AND ( i.publish_down = '0000-00-00 00:00:00' OR i.publish_down >= '2017-10-31 14:53:20' ) AND c.id IN (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17) AND i.featured = 1 AND i.id > 25000 ORDER BY i.featured DESC, i.created DESC LIMIT 0, 10;

The above query typically runs on the homepage of the high traffic Joomla site, and is used to return a list of featured K2 items belonging to (K2) categories 1 to 17. The query, as you may have already noticed, is already optimized. However, it is slow, and this is because of the following:

c.id IN (1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17)

As you can see in the above, the query uses MySQL’s IN, which is very expensive. Well, not as expensive as buying a whole watermelon in Iqaluit, Nunavut (this is the in North Pole, in case you just don’t have time to google it), but still, really expensive. The reason why the query uses IN is to restrict the search to a number of specific categories. But, is this really necessary?

In the case of this website, it was not. First, all of the K2 categories on the Joomla site consisted of the 17 categories listed above, and second, even if there were other K2 categories than the ones listed above, the filtering on the featured field was enough (the featured field is mainly used to filter the items to be displayed on the homepage). So, we logged in to the backend of the website, and we went to the Home menu item (e.g. the menu item where the Home flag is set), and we clicked on options, and we removed all the categories from Select Categories. We saved the menu item and then we cleared the cache. We then verified that the website was showing the same content that it was displaying before on the homepage (just to make sure that we didn’t break something in the process).

After doing the above, we monitored the slow query log for that website for 24 hours, and, guess what, there were no more slow queries! Hooray!

So, in case you are having some performance issues on your K2 powered website, then make sure you are not filtering on categories when you don’t need to do so (if you are displaying only featured items on the homepage, then it is almost a certainty that you don’t need to have any filtering on the categories). If the K2 performance issues are caused by something else, then please contact us. We are experts in Joomla optimization, our work is super clean, and our fees are super affordable!

“Fatal error: Class ‘Joomla\CMS\Menu\Tree’ not found” Error After Failed Joomla Update

This past weekend, a client of ours from Kuwait emailed us and told us that the backend of his Joomla website is displaying a blank page. He told us that the login page to the backend was working, but, as soon as you login, you will see a blank page. He also told us that the problem happened after updating the obRSS extension.

Naturally, the first thing that we did was to disable the obRSS system plugin by renaming its folder under the plugins/system folder, but, unfortunately, that didn’t fix the problem, so the problem must have been caused by something else.

We then enabled error reporting on the Joomla website by setting the value of the $error_reporting variable to maximum (in the global configuration.php file), but we still saw a blank page. This meant that the “@” operator was used and/or output buffering was enabled somewhere, so we tried to disable output buffering through various means, but that didn’t work. We still saw a blank page.

After our 2 failed attempts to unveil the actual error, we decided to debug the issue the good old way. We renamed the whole modules folder (under the administrator folder) to modules_old and we tried to load the backend, and this time, it did work (albeit with no modules whatsoever). This meant that the problem was caused by one of the administrator modules. So we renamed the modules_old folder back to modules, and then we renamed the folders (modules) inside that folder one by one, and we tested the website after each rename. Once we renamed the mod_menu folder to mod_menu_old, the problem was no more! This was weird, we thought, since the problem turned out to be with a core module (we thought that this was caused by a 3rd party module/plugin).

After discovering that the problem was caused by the mod_menu folder, which is located under the administrator/modules folder, we needed to display the actual error. So, we added the following 3 lines to the very beginning of the menu.php file (which is located under the administrator/modules/mod_menu folder):

ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);

After doing the above, we loaded the backend of the Joomla website, and this time we saw an error instead of a blank page. It was this one::

Fatal error: Class 'Joomla\CMS\Menu\Tree' not found in /home/[cpanel-user]/public_html/administrator/modules/mod_menu/menu.php on line 63

Hmmm! The above error felt like it was caused by an incomplete Joomla 3.8.1 update, as it seems that the backend menu module was trying to use dependencies from a Joomla 3.8 instance despite the fact that the Joomla website was running 3.7.0. So, we renamed the libraries folder to libraries_old, and recopied the libraries folder from a fresh copy of Joomla 3.7.0, but that didn’t fix the problem.

So we thought, maybe the problem was only in the aforementioned menu.php file, so we copied that file from a fresh Joomla 3.7.0 install, and we loaded the backend of the site. This time, we saw a different error, it was this one:

Fatal error: Call to undefined method ModMenuHelper::getComponents() in /home/[cpanel-user]/public_html/administrator/modules/mod_menu/preset/enabled.php on line 297

So, we did the same thing for the enabled.php file (we copied it from a fresh Joomla 3.7.0) install. We then reloaded the backend, and yet another different error was displayed:

Fatal error: Call to undefined method JAdminCssMenu::getTree() in /home/[cpanel-user]/public_html/administrator/modules/mod_menu/tmpl/default.php on line 17

At this point, we were no longer having fun. So we copied the whole administrator/mod_menu folder from a fresh copy of Joomla 3.7.0 to the current site (after deleting the old mod_menu folder), and this time, it worked! There were no errors being displayed.

Despite fixing the problem, we didn’t feel satisfied, because we felt that the website was in an unstable state. From our experience, a failed Joomla update doesn’t only affect one folder. So, we asked for permission from the client to update the website, and we were granted that permission immediately.

We initiated the Joomla update by clicking on the Update Now button on the home screen of the backend of the Joomla website (of course, we backed up both the filesystem and the database of the website before proceeding), and, instead of having a smooth update, we saw the following error:

Error: Could not open /home/[cpanel-user]/public_html/cache/index.html for writing.

So, we loaded the backend again, and this time, we saw a blank page. We immediately suspected that the blank page was caused by the administrator menu module, and disabling that module by renaming the mod_menu folder (under the administrator folder) to mod_menu_old confirmed our suspicions. Not only that, this failed update allowed us to understand the original cause of the problem: the client tried to update the Joomla website, and it failed on him, but, it did that after updating the administrator menu module to its Joomla 3.8.1 code, which made the module incompatible with the current website, thus triggering the error.

So, we reverted the administrator menu module to Joomla 3.7.0 (again), and we checked the permissions on the index.html file (which is located under the cache folder). They were the following:

-rwxrwxrwx 1 root root 0 Apr 26 2017 index.html*

Obviously, the permissions and the ownership of the file were both wrong, so we fixed them by issuing the following 2 commands in the shell:

chmod 644 index.html
chown [cpanel-user].[cpanel-user] index.html

We then reattempted the update and this time it worked! The website was updated to Joomla 3.8.1 and was working without a hitch. Hooray!

We hope that you found this post useful and that it helped you resolve your problem. If it didn’t, or if you need help with the implementation, then please contact us! Our fees are affordable, our work is professional, and we will be your true Joomla friends!

What Happens if Joomla Does Not Close MySQL Database Connections

At itoctopus, we have a strong passion for optimizing Joomla websites, that’s why we always research new ways that may improve their performance. We mostly focus on optimizing the database because sometimes a simple database optimization can have a huge impact on the overall performance of the Joomla website. This time it was no different.

However, the database optimization that we attempted wasn’t about optimizing a query, or indexing a field, or modifying the my.cnf file (the MySQL global configuration file). It was about answering the million dollar question that many have asked but no one has given a straight answer to: “Does closing the connection to the MySQL database cause a performance hit”? Many have provided theories and opinions about this, but none (as far as we know) has given an answer based on an actual experiment. We decided to be the first ones to provide a real, experiment-based answer to this question, not only because it hasn’t been answered before, but also because we were very curious.

Joomla does explicitly close the connection to MySQL in the mysqli.php file which is located under the libraries/joomla/database/driver folder. So we decided to do the following test:

  • Run the following ab (Apache Benchmark) on an unmodified Joomla 3.8.1 website (with test data installed) with n concurrent connections, where n is 10, 20, 30, 40, and 50:

    ab -c n -t 10 "http://www.joomla381.com/"

  • Remove the following line from the aforementioned mysqli.php file (note: the line is in the disconnect function):

    mysqli_close($this->connection);

  • Run the above ab test again on the modified Joomla website, where the MySQL connection is no longer closed.

Unlike our comparison between the performance of Joomla 3.7.5 and Joomla 3.8.1, we didn’t get any exciting data. The data was more or less the same. A few milliseconds here and a few milliseconds there. Nothing of significance was observed. There was no constant advantage for one over the other.

In fact, we thought (hoped?) that not closing the connection would provide a boost (albeit a negligible one) to the performance of the Joomla website, and this is because there is overhead in opening and closing MySQL connections, but it seems that such an advantage (if it really existed in the first place) was offset by something else.

So there you have it! There is no advantage in not closing the connection to the MySQL database on a Joomla website. If you have a different experience, then please comment below. If you need help optimizing your Joomla website, then please contact us. We are experts in Joomla optimization, our work is clean, and our fees are right!

Joomla Has a Mystery “ordering” Form Field Type

Yesterday evening (or very early in the morning today), we updated the Joomla website of a new client from Joomla 3.1.5 to Joomla 3.7.5 (we are waiting for 3.8.2 to be released to update to 3.8). The update was really nothing out of the ordinary: we did see some issues but we fixed them immediately as all of these issues were déjà vu. We were going to email the client and tell them that we were done, but, as we were about to do that, we noticed a weird error on a custom component, which was more or less like the core Joomla content component, but, it had departments instead of articles. The problem was that when we clicked on a department to edit it (or when we tried to create a new department), we saw the following error:

You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ‘WHERE `catid` = 0 ORDER BY ordering’ at line 3

We analyzed the view, and we were able to narrow down the problem to the following field in the department.xml file under the administrator/components/com_department/models/forms folder:

<field
	name="ordering"
	type="ordering"
	class="inputbox"
	label="JFIELD_ORDERING_LABEL"
	description="JFIELD_ORDERING_DESC"
/>

Naturally, we searched for the form field of type ordering defined at the component level, and we found it, but we didn’t find any mention of catid anywhere in the extension, so the fatal query wasn’t coming from that field. Strangely though, when we removed the field from the XML file, everything worked as it should (but, of course, without the ordering).

So, we enabled debugging in the global configuration of the Joomla website by setting the value of Debug System to “Yes” under System -> Global Configuration -> System. We then loaded the page and we immediately discovered the cause of the problem: the ordering field was being loaded from another location. It was being loaded from the ordering.php core file which is located under the libraries/cms/form/field folder (note that in Joomla 3.8+, the ordering field is located in the OrderingField.php file, which in turn is located under the libraries/src/Form/Field folder). But, what is that field? And why isn’t documented on the Joomla official website?

Further investigation revealed that this field was introduced in Joomla 3.2.0, and that it is used in 3 extensions: com_banners, com_contact, and com_newsfeeds (by the way, we just noticed, why doesn’t Joomla have some standard when it comes to singular/plural naming in its core extensions; why isn’t com_banner and com_newsfeed instead of com_banners and com_newsfeeds?). The core ordering field does exactly what you expect it to do: it displays a drop down of all the content items belonging to the category of the current content item, which allows you to set the latter’s ordering. The field expects the category name of the current content item to be called catid. If it doesn’t find such field, then it crashes, which fully explains the problem that we were having on the client’s site.

Here’s an example usage of the ordering field from the com_banners extension:

<field
	name="ordering"
	type="ordering"
	label="JFIELD_ORDERING_LABEL"
	description="JFIELD_ORDERING_DESC"
	table="#__banners"
/>

You can see that all that you need to pass as a parameter is the table field, which is #__banners in the case of the com_banners extension (note that in previous versions, such as Joomla 3.2.0, you needed to pass the content_type, such as com_banners.banner).

But why wasn’t the “ordering” Joomla form field type documented?

We’re not really sure but we think it was just that the Joomla documentation team forgot about it (we will report this omission to the Joomla core team) – this is plausible because Joomla officially added a whopping 8 fields in Joomla 3.2 (so the ninth one could have easily fell through the cracks). It might also be due to the fact that this field type feels unfinished and non-generic. For example, why is catid hardcoded in the field? Still, the ordering field has its merits, especially for extensions taking advantage of the #__content and the #__categories tables.

So, what did we do to fix the problem?

We fixed the problem by appending the word department to every instance of the word ordering for that field type. Here’s how:

  • We renamed the file ordering.php which is located under the administrator/components/com_department/models/fields to departmentordering.php.
  • We then opened the departmentordering.php file, and we changed the following line:

    class JFormFieldOrdering extends JFormField

    to:

    class JFormFieldDepartmentOrdering extends JFormField

    We also changed the following line:

    protected $type = 'Ordering';

    to:

    protected $type = 'DepartmentOrdering';

  • Finally, we opened the department.xml file which is located under the administrator/components/com_department/models/forms folder and we changed the following:

    <field
    	name="ordering"
    	type="ordering"
    	class="inputbox"
    	label="JFIELD_ORDERING_LABEL"
    	description="JFIELD_ORDERING_DESC"
    />

    to:

    <field
    	name="ordering"
    	type="DepartmentOrdering"
    	class="inputbox"
    	label="JFIELD_ORDERING_LABEL"
    	description="JFIELD_ORDERING_DESC"
    />

Once we did the above, the problem was solved! Woohoo!

Now, if you, our dear reader, are having the same problem and you need help with implementing the solution, then fear not, we are your Joomla super heroes! Just contact us and we’ll fix your website quickly, professionally, and affordably!

Comparing the Performance of Joomla 3.8.1 to Joomla 3.7.5

Note: This a very lengthy post. So, it’ll be a good idea to read it first thing in the morning when your mind is fresh and while you’re having your wake up coffee (or tea, or milk, or cold water, or nothing).

Another note: This is a very technical and a very advanced post. It is mainly aimed for developers, but non-developers can still enjoy it nonetheless, or can just scroll down to the bottom of this post for the chart.

So far, we haven’t upgraded any of our customers to Joomla 3.8.x, and this is because we learned our lesson a few years ago to wait a couple of months before updating to the latest version of Joomla (unless, of course, the update is a major security patch). However, we are very close followers of the Joomla community, and we have read and heard many comments about the bad performance of Joomla 3.8 when it is compared to Joomla 3.7. We did communicate these concerns to the Joomla developers, who discovered that some events were triggered twice in Joomla 3.8.0 (leading to a major performance issue on large Joomla sites), and who fixed that problem in 3.8.1.

So, is Joomla 3.8.1 faster or slower to Joomla 3.7.5? In this post, we decided to benchmark Joomla 3.8.1 against Joomla 3.7.5, in order to provide a definitive answer to the question.

The Setup

We used an idle dedicated server for our test. The server had the following technical specifications:

  • Processor: Intel Xeon E3-1271 v3 Quad-Core
  • Memory: 16 GB DDR3 SDRAM (SDRAM, in case you’re wondering, stands for Synchronous Dynamic Random Access Memory)
  • Disk: 250 GB SSD Drive

The following software was installed on the server:

  • Operating system: CentOs 6 – 64Bit
  • Web server: Apache 2.4
  • Database server: MySQL 5.5
  • PHP version: PHP 5.6

Since we didn’t have any benchmarking tool on the server, we installed ab, the Apache HTTP server benchmarking tool, which is developed by the Apache Software Foundation. Here’s how we installed it

  • We edited the yum.conf file which is located under the /etc folder to remove httpd from the exclusion list. We opened the file using vi:

    vi /etc/yum.conf

    From the exclude line (which is the second line in the file), we removed the httpd* entry, to allow the install of the ab tool through yum. We saved the file and exited vi.

  • We then issued the following command in order to ensure that we can now install ab through yum:

    yum provides /usr/bin/ab

    (Note: If you see No matches found when you issue the above command, then try contacting your host).

  • Once we were sure that we were able to install ab, we installed it using the following command:

    yum install httpd-tools

Now that we have installed ab, te next step was to create 2 cPanel accounts, joomla375.com and joomla381.com (this was easily done through WHM). As you might have guessed, the first account was created in order to install a Joomla 3.7.5 website, while the latter was used for a Joomla 3.8.1 website. But, before installing Joomla on either account, we had to modify the hosts file on our PC and on the server so that these domains resolve to the test server. For our PC, we modified the hosts file which is located under the C:\Windows\System32\drivers\etc folder to include the following 2 lines:

[ip-address-of-the-server] joomla375.com www.joomla375.com
[ip-address-of-the-server] joomla381.com www.joomla381.com

For the server, we added the same lines above in the /etc/hosts file.

At this point, we were ready to install Joomla. We installed Joomla 3.7.5 on joomla375.com and Joomla 3.8.1 on joomla381.com (we will spare you the boring installation details), and – we chose to include the blog test data for both. joomla375.com and joomla381.com ended up being identical sites.

We opened the file my.cnf file under the /etc folder and we added the following 2 lines immediately after [mysqld]:

general_log = on
general_log_file=/var/lib/mysql/all-queries.log

We then restarted MySQL. In case you’re wondering, the above 2 lines will log any query that hits the database server to the all-queries.log file (located under /var/lib/mysql/ folder).

The Benchmarking

Typically, benchmarking is done by issuing a number of concurrent connections to the website for a specific amount of time using a benchmark tool such as ab, then examining the results and comparing them to a different website. However, since we are working on 2 identical websites using the same CMS with a different version, we thought that it’s a good idea to add another metric to the test, which is the size of the query file generated by a page load (or a number of pages loads). In other words, we load the homepage of each Joomla website, with the general_log on, and we compare the size (in bytes) of the all-queries file generated by each Joomla website to the one generated by the other. We will consider that smaller is better (we agree that it is not an accurate benchmark, since the complexity of a query is not always proportional to its string length).

We started with the all-queries benchmark first. Here’s how we did it:

We first loaded the homepage of the Joomla 3.7.5, and we examined, the size in bytes, of the all-queries.log file, and it was 21473 bytes. We emptied the all-queries.log file, and we did the exact same test on the Joomla 3.8.1 website. The size of the file was 18552 bytes. For us, it was a shocking surprise, because we expected the complete opposite. We didn’t expect Joomla 3.8.1 to have about 3 KB less queries than Joomla 3.7.5, so naturally, we were curious. We compared the 2 files and it turned out that Joomla 3.8.1 got rid of all the unnecessary (and never used) references to the #__content_rating table. We did mention the unnecessary references to the #__content_rating table before, and it seems that someone from the Joomla core team was listening then. Other than the removal of the #__content_rating table, we didn’t really find much difference between the 2 files. They were almost identical.

Our next test scenario was slightly more elaborate, as it consisted of doing the following (in the exact sequence below) on the 2 sites:

  • Logging in to the Joomla website.
  • Adding a user with default privileges.

  • Adding an article to the Blog category.

  • Viewing the homepage.

In this test, the results were more in line with our expectations: the size of the all-queries.log file was 161394 bytes for Joomla 3.7.5 and 209505 bytes for Joomla 3.8.1, which amounted to almost exactly a 30% increase in the size of the query file in Joomla 3.8.1. Again, we were curious as to what was different, and so we compared the 2 files. This time, we discovered something very interesting…

The all-queries.log file of the Joomla 3.8.1 website was littered with the following query:

SELECT COUNT(`extension_id`) FROM `#__extensions` WHERE `element` = '[com_extension]' AND `type` = 'component'

com_extension was the name of a component, such as com_content, com_menu, etc…. There were hundreds of these weird queries (375 to be exact) – many were repeated! For example, the following query…

SELECT COUNT(`extension_id`) FROM `#__extensions` WHERE `element` = 'com_menu' AND `type` = 'component'

…was literally 45 times in the log file.

These 375 queries didn’t exist in Joomla 3.7.5 (375 queries didn’t exist on Joomla 3.7.5, is it really just a coincidence, or were the developers trying to tell us something?), and these 375 queries resulted in the 30% increase in the size of the log file.

Now, this query logging method is not scientific, and we didn’t really expect to learn much from it, but we did. We did learn that there is a huge overhead (about 50 additional queries per page) in Joomla 3.8.1, and it’s all because of these weird queries…

The next step was to use the classical method for benchmarking, which is the ab method. So, we issued the following 2 commands in the Linux shell:

ab -c 10 -t 10 "http://www.joomla375.com/"

which returned the following:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.joomla375.com (be patient)
Finished 737 requests


Server Software:        Apache/2.4.23
Server Hostname:        www.joomla375.com
Server Port:            80

Document Path:          /
Document Length:        18052 bytes

Concurrency Level:      10
Time taken for tests:   10.000 seconds
Complete requests:      737
Failed requests:        0
Write errors:           0
Total transferred:      13676509 bytes
HTML transferred:       13304324 bytes
Requests per second:    73.70 [#/sec] (mean)
Time per request:       135.688 [ms] (mean)
Time per request:       13.569 [ms] (mean, across all concurrent requests)
Transfer rate:          1335.57 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    56  133 139.6    115    1482
Waiting:       56  132 139.6    115    1482
Total:         56  133 139.6    115    1482

Percentage of the requests served within a certain time (ms)
  50%    115
  66%    124
  75%    131
  80%    135
  90%    146
  95%    162
  98%    183
  99%   1306
 100%   1482 (longest request)

…and…

ab -c 10 -t 10 "http://www.joomla381.com/"

…which returned the following:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.joomla381.com (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Completed 25000 requests
Completed 30000 requests
Completed 35000 requests
Completed 40000 requests
Completed 45000 requests
Completed 50000 requests
Finished 50000 requests


Server Software:        Apache/2.4.23
Server Hostname:        www.joomla381.com
Server Port:            80

Document Path:          /
Document Length:        328 bytes

Concurrency Level:      10
Time taken for tests:   2.643 seconds
Complete requests:      50000
Failed requests:        0
Write errors:           0
Non-2xx responses:      50000
Total transferred:      28000000 bytes
HTML transferred:       16400000 bytes
Requests per second:    18918.25 [#/sec] (mean)
Time per request:       0.529 [ms] (mean)
Time per request:       0.053 [ms] (mean, across all concurrent requests)
Transfer rate:          10345.92 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:     0    0   0.1      0       1
Waiting:        0    0   0.1      0       1
Total:          0    1   0.1      0       1
ERROR: The median and mean for the total time are more than twice the standard
       deviation apart. These results are NOT reliable.

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%      1 (longest request)

The first command was used to benchmark the Joomla 3.7.5 website, and the second command was used to benchmark the Joomla 3.8.1. So, which one did better?

Before examining the results of the ab command (which are very interesting), let us dissect and explain the ab command line that we issued:

ab -c 10 -t 10 "http://www.joomla381.com/"

The above means that we are sending 10 concurrent (or simultaneous) connections (-c 10) for a maximum of 10 seconds (-t 10) on the website http://www.joomla381.com/ (Note: adding -k to the above command will force the use of the HTTP KeepAlive feature).

Going back to the results, at first glance, it is very obvious that Joomla 3.8.1 did much, much better than Joomla 3.7.5. In fact, Joomla 3.8.1 was able to process 50,000 requests in 2.643 seconds, while Joomla 3.7.5 only processed 737 requests in 10 seconds. The time per request for Joomla 3.8.1 was a a very impressive 0.529 milliseconds, while that of Joomla 3.7.5 was a very slow 135.688 milliseconds. So, if we were to end this article now and we were to use this benchmark data, then we can conclude that Joomla 3.8.1 is 257 times faster than Joomla 3.7.5. This is just too good to be true – more like unbelievable (and rightly so, as you will learn later in this post). So we closely examined the above numbers and we noticed that there was some kind of cheating going on. You see, while the Joomla 3.7.5 website responded with 18052 bytes (this is the Document Length), Joomla 3.8.1 responded with a mere 328 bytes. Clearly, the Joomla 3.8.1 website wasn’t giving the full response.

As you might have guessed, we investigated further…

So, we grabbed the Joomla 3.8.1 website using wget with the following command…

wget http://www.joomla381.com/

…and we got the following response:

Resolving www.joomla381.com... [ip-address]
Connecting to www.joomla381.com|[ip-address]|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden

Aha – the website was returning a 403 error, and then it hit us, we denied, in the .htaccess file of the Joomla 3.8.1 website, all IPs from accessing it with the exception of our IP; so we needed to add the IP of the server as well. The Joomla 3.7.5 website didn’t have the problem because the server IP was already authorized in an allow statement in the .htaccess file.

So, it wasn’t really a conspiracy or cheating or foul play or anything (it’s always easy to blame the poor Joomla development team for anything), it was just us being tired. So we decided to continue our experimentation the next morning.

The next morning (well, at 3 AM to be exact), we continued our work. The first thing that we did was to add the server IP to the .htaccess file of the Joomla 3.8.1 website, and then we re-executed the following ab statement in the shell:

ab -c 10 -t 10 "http://www.joomla381.com/"

This time, it returned totally different results:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.joomla381.com (be patient)
Finished 714 requests


Server Software:        Apache/2.4.23
Server Hostname:        www.joomla381.com
Server Port:            80

Document Path:          /
Document Length:        18691 bytes

Concurrency Level:      10
Time taken for tests:   10.038 seconds
Complete requests:      714
Failed requests:        0
Write errors:           0
Total transferred:      13705944 bytes
HTML transferred:       13345374 bytes
Requests per second:    71.13 [#/sec] (mean)
Time per request:       140.583 [ms] (mean)
Time per request:       14.058 [ms] (mean, across all concurrent requests)
Transfer rate:          1333.45 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    58  137 146.1    117    1530
Waiting:       58  137 146.1    117    1530
Total:         58  137 146.1    118    1530

Percentage of the requests served within a certain time (ms)
  50%    118
  66%    127
  75%    134
  80%    138
  90%    153
  95%    168
  98%    187
  99%   1332
 100%   1530 (longest request)

Now it’s more like it. The document length was 18691 bytes, which is very close to the 18052 bytes returned by the Joomla 3.7.5 website (by the way, it would be interesting to know why there is a 639 bytes difference between the 2 – but it’s really outside the scope of this post). The number of completed requests is 714 requests, with a mean time of 140.583 milliseconds. The Joomla 3.7.5 website, in case you don’t feel like scrolling up a bit, handled 737 requests with a mean time of 135.688 milliseconds.

The Chart

At this point, the whole thing became very interesting. We had meaningful and close results. So, we decided to play with the number of connections: we issued a series of ab commands on each website with increasingly higher number of connections (e.g. we changed -c 10 to -c 20, -c 30, etc… – note that we waited a few minutes after each command for the load to return back to 0), and we got this beautiful chart (probably what you have been waiting for since the beginning of this post):

Joomla 3.7.5 vs Joomla 3.8.1

Figure 1: Joomla 3.7.5 vs Joomla 3.8.1

You can see in the above chart 2 things: 1) Joomla 3.7.5 consistently performs better than Joomla 3.8.1, but only slightly, and 2) the 2 lines diverge more with the number of connections. In fact, Joomla 3.8.1 is 3% slower than Joomla 3.7.5 with 10 concurrent connections, but with 50 concurrent connections, it is 6% slower.

So, there you have it. Joomla 3.8.1 is slower than Joomla 3.7.5, and we’re totally not lying with stats charts (please trust us!). In fact, you can conduct the same test yourself and you will likely get similar results.

We hope that you found this post informative and that you had fun reading it. If you have any questions about it, or if you need help optimizing your Joomla 3.8.1 website, then all you need to do is to contact us. Our fees are affordable (and haven’t increased for the past 7 years), our work is clean, and we are the friendliest Joomla developers in the Milky Way (unless, of course, Mars turns out to be populated by extremely friendly Joomla developers).

“Application Instantiation Error: No database selected” Joomla Error

A new client emailed us that he was seeing the following error on his Joomla website:

Error displaying the error page

He told us that he started seeing the above error when the hosting company moved his website from one server to another. He told us that the hosting company debugged the issue for a long time, but couldn’t determine the problem.

So, we logged in to the cPanel account of the website, and then we went to the File Manager, and from there, we edited the configuration.php file and changed the value of $error_reporting from default to maximum (we couldn’t do that through the backend of the Joomla website since both the backend and the frontend were showing the same error).

When we visited the website again, it displayed the following error:

Error displaying the error page: Application Instantiation Error: No database selected

Aha, now we have a real, meaningful error, and we know that it has to do with the database. But, are the connection parameters incorrect? No, because if they were, then we would see the following error:

Error displaying the error page: Application Instantiation Error: Could not connect to MySQL.

So, we know for sure that the database host, username, and password (these are all set in the configuration.php file) are all correct. But, is the database name correct? So, we checked the name of the database in the configuration.php and we matched it to the one in phpMyAdmin and it did.

We were confused – we were confident that the database parameters were correct, yet we were sure that the problem was with the database parameters (yes, we know, it doesn’t make any sense!). So, we went back to the cPanel homepage, we clicked on the MySQL databases icon, we created another database user, and we assigned that user to the database (we granted the user all the privileges). We then modified the configuration.php values for $user and $password to match those of the user that we just created. We loaded the website and this time it worked. But why wasn’t it working for the old user?

We investigated the issue by comparing the user that we have created and the previous user, and it didn’t take us long to see the different: the old user didn’t have any privileges on the database. It seems that the user was created by the hosting company, but it was not assigned any privileges to the database. So, under the Add User To Database section on the MySQL Databases page in cPanel, we selected the old user under User, and then we selected the Joomla database under Database, we then clicked on the Add button, and on the next page, we clicked on the checkbox ALL PRIVILEGES to select all possible database privileges, and finally we clicked on Make Changes at the bottom. We reverted back the configuration.php to what it was and, unsurprisingly, the website worked!

So, if you have the same problem on your website, then make sure that the right privileges are assigned to the database user on the Joomla database. If you have already done that and it didn’t fix the problem, or if you need help doing that, then please contact us. Our fees are right, our work is clean, and our Joomla experience is unquestionable!

A Super Elegant Method to Check if a Joomla User is a Super User

A few weeks ago, we wrote a post in which we admitted that we haven’t discovered fire. This morning, however, we had a new discovery akin to discovering fire, and it was a very clean, very elegant, very lovely method to check if a user is a super user. It all happened by coincidence when were checking the contents of the UserHelper.php file which is located under the libraries/src/User folder. We believe that our discovery is a defining moment in the history of mankind!

We know you’re anxious to know what the method is, so, here’s the code:

JUserHelper::checkSuperUserInUsers(array('5'));

The above checks if 5 is the ID of a super user. If it is, then the function will return true, if not, then it will return false.

What happens if you supply an array of users to the function?

As you can see from the above code, the checkSuperUserInUsers static method accepts an array of users (not just one user), so what happens if you pass an array of users to the function? Well, the function will return true as soon as it detects a user in the array that is a super user. If there are no super users in the array, then the function will return false.

Why does this method exist?

The checkSuperUserInUsers is mainly used to block batch actions on super users when the user performing the action is not a super user. For example, if a “non super user” selects a group of users in the Joomla backend, and clicks on the Delete button on the top, then Joomla will not allow the deletion when one or more selected users are super users. The function checkSuperUserInUsers is used to do the checking.

We hope that you enjoyed our little post this Saturday morning. If, by any chance, you have a Joomla development project, and you need help with it, then please contact us. Our prices are right, our code is clean, and we have over a decade of Joomla experience.

“Call to undefined method JApplicationSite::isClient()” when Updating to Joomla 3.8

A new client approached us yesterday and told us that they were seeing a blank page on the frontend and on the backend of their Joomla website after updating it to 3.8.0. Since a blank page is a sign of a fatal error, we set the error reporting on the client’s site to “maximum” in the configuration.php file, and then checked the frontend of the website, which was displaying the following:

Fatal error: Call to undefined method JApplicationSite::isClient() in /home/[cpanel-user]/public_html/plugins/system/languagefilter/languagefilter.php on line 94

The backend of the website was displaying the following error:

Fatal error: Call to undefined method JApplicationAdministrator::isClient() in /home/[cpanel-user]/public_html/plugins/system/languagefilter/languagefilter.php on line 94

(Note: Enabling debugging displayed the following error on the backend and on the fronted of the Joomla website: Fatal error: Call to undefined method JProfiler::setStart() in /home/[cpanel-user]/public_html/administrator/index.php on line 45)

If you look closely at the above errors, you will notice that the problem is that 2 basic Joomla classes, JApplicationSite and JApplicationAdministrator, are not being loaded. But why?

We investigated the problem very thoroughly. We looked at all the files that were used to load core classes or support the loading of core classes, notably, the following 2 files:

  • The file loader.php which is located under the libraries/cms/class/ folder.
  • The file ClassLoader.php which is located under the libraries/vendor/composer folder.

We couldn’t find anything.

We then thought it could be a problem with the PHP version that the client was using, which was PHP 5.4, and so we switched to 5.6, and then 7.0, and then 7.1. None worked!

We then started thinking about the hosting environment, could it be? Well, the client was using GoDaddy, which should cause any decent developer to become a bit skeptical at the very least (in fact, we could just blame GoDaddy for the heat wave we’re having right now in Montreal and everyone would believe us!). So, we copied the website over to one of our servers, and we tested it there. Same exact problem!

Eventually, we decided to copy a fresh copy of Joomla 3.8 onto the website. So, we downloaded Joomla 3.8, we then removed the images and the templates folder from the downloaded zip file, we extracted the zip file onto the website, and we tried loading the website, still, the same error.

Then it suddenly hit us, what if the problem is similar to this other problem that we resolved a while ago which was caused by Joomla loading old library files instead of the new ones? So, we renamed the existing libraries folder to libraries_old, and we copied a fresh copy of the libraries folder onto the website (from the Joomla 3.8 zip file that we just downloaded) and then we tested it. This time it worked!

Aha! So, the problem was caused by the wrong library file(s) being loaded? But which one(s)? We were determined to find out the answer to our question, so, we printed out all the included files on the fixed site, and then, we reverted back and printed out all the included files on the broken site, and we compared the list of files.

There were, in fact, many, many files that were included from the wrong places on the broken Joomla website. For example, the factory.php file was included from the libraries/joomla folder instead of the libraries/src folder. There were also many files included from the libraries/cms folder, and these files were included from folders that should no longer exist under the libraries/cms folder. In fact, the libraries/cms folder in Joomla 3.8 has only 3 folders inside it (Joomla 3.7.5 has 30 folders inside the libraries/cms folder).

It seems that the client has either updated the site manually (e.g. by overwriting the files from a fresh Joomla install), or the client has updated the site from the backend but from an old Joomla website where the update process does not include removing these old library files/folders. In any case, we think the problem is really caused by Joomla, which should take into consideration any update scenario. The problem is also ironic, since it highlights the fact that there is a huge change in the file structure in Joomla 3.8, completely contradicting a statement by a core Joomla developer that the “the difference from 3.7.5 is relatively minor”.

So, if you are seeing the Call to undefined method JApplicationSite::isClient() error on your Joomla website, then try renaming the libraries folder to libraries_old and then copying the libraries folder from a fresh Joomla 3.8 install. If it doesn’t work, or if you are seeing a different error, then please contact us. We are always ready to help, our fees are very reasonable, and we really love working on Joomla websites!

How We Integrated Joomla’s “Email this Link to a Friend” with HubSpot

Note: This integration consists of a core override. As usual, please keep in mind that core overrides can lead to stability issues and can be wiped out with a future Joomla update.

A client of ours asked us to integrate Joomla’s “Email this Link to a Friend” (or “Send Article to a Friend”) with HubSpot. What they wanted to do was to add the person sending the article as well as the person receiving it as contacts. As you already know, when someone sends an article to a friend, he is asked to fill in his name, his email, and his friend’s email. So, in other words, we will have the name and the email of the first contact (the person who sends the article), while we will have only the email of the second contact (the person who receives the article).

The first thing that we did was creating the form in HubSpot. The form contained 3 fields: “Email”, “First Name”, and “Last Name”. The internal field names for those fields were “email”, “firstname”, and “lastname”, respectively. Only the email was set to required. Once we created the form, we clicked on the Embed tab on the left, and we saw the following in the textarea just under Copy the snippet below onto your site:

<!--[if lte IE 8]>
<script charset="utf-8" type="text/javascript" src="//js.hsforms.net/forms/v2-legacy.js"></script>
<![endif]-->
<script charset="utf-8" type="text/javascript" src="//js.hsforms.net/forms/v2.js"></script>
<script>
  hbspt.forms.create({ 
    portalId: '111111',
    formId: '1a1a1a1a-2b2b-3c3c-4d4d-5f5f5f5f5f5f'
  });
</script>

We then took note of the value of portalId (which was 111111) and formId (which was 1a1a1a1a-2b2b-3c3c-4d4d-5f5f5f5f5f5f) in the code above. (Note: These are not the real values and you should replace portalId and formId with your form’s values).

After creating the form and taking note of the values of portalId and formId, we moved over to the fun stuff: it was time for sending the contacts from the Send Article to a Friend Joomla form to HubSpot. So, we opened the file controller.php which is located under the components/com_mailto folder, and we added the following function at the very beginning of the MailtoController class:

private function addToHubSpot($email, $name=''){
  //note - you will need to replace "111111" with your portalId, and you will need to replace "1a1a1a1a-2b2b-3c3c-4d4d-5f5f5f5f5f5f" with your formId.
  $url = 'https://forms.hubspot.com/uploads/form/v2/111111/1a1a1a1a-2b2b-3c3c-4d4d-5f5f5f5f5f5f';

  $pageURL = JUri::current();
  $pageName = "Send Article to a Friend";
  
  $name = trim($name);
  if (!empty($name)){
    $arrName = explode(' ', $name);
    if (count($arrName) > 0){
      $firstname  = $arrName[0];
      array_shift($arrName);
      $lastname = implode(' ', $arrName);
    }
    else{
      $firstname = $arrName[0];
      $lastname = $arrName[0];
    }
  }
  
  $fields = array(
    'firstname' => $firstname,
    'lastname' => $lastname,
    'email' => $email
  );
  $strFields = http_build_query($fields);

  jimport('joomla.user.helper');
  $randomPassword = JUserHelper::genRandomPassword(32);
  
  $hsContext = array(
      "hutk" => $randomPassword,
      "ipAddress" => "192.168.1.12", 
      "pageUrl" => $pageURL,
      "pageName" => $pageName
  );
  $strContext = json_encode($hsContext);
  $strContext = urlencode($strContext);
  $strFields = $strFields.'&hs_context='.$strContext;
  
  $ch = curl_init($url);
  curl_setopt($ch, CURLOPT_POST, true);
  curl_setopt($ch, CURLOPT_POSTFIELDS, $strFields);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($ch, CURLOPT_TIMEOUT, 2); //wait a maximum of 2 seconds
  curl_setopt($ch, CURLOPT_HTTPHEADER, array(
      'Content-Type: application/x-www-form-urlencoded'
  ));  
  $result = curl_exec($ch);
}

Then, just before $this->display(); near the very end, we added the following code:

//add the sender to HubSpot
$this->addToHubSpot($from, $sender);
//now add the person receiving the email only if it is different than the person sending the email
if ($from != $email){
  $this->addToHubSpot($email, '');
}

As you can see in the above code, we are only adding the receiver's email if the email of the person sending it is different from that of the receiver. This is because many people send articles to themselves.

Now, after doing the above, we filled in an "Email this Link to a Friend" form, and to our pleasant surprise, it worked as it should: it made 2 form submissions in HubSpot and created 2 contacts! Hooray!

We hope that you enjoyed this post. If you need help with this integration or with any HubSpot integration on your Joomla website, then please contact us. We have implemented many HubSpot integrations on many Joomla websites, our fees are super affordable, and our work is super clean!

7 Reasons Why Your Joomla Website Response Time Is Suddenly High

We often get calls/emails from clients stating that the response time of their Joomla websites is suddenly high. They say that the day before everything was fine, and all of a sudden in the morning, the website became super slow. Of course, the first thing that we ask them is: “Has anything changed between yesterday and today?”, and their answer is typically “Not that I know of”. But, when we work on the problem, we discover that something did change, and that “something” was the cause of the problem. Since we have worked on many of these websites, we have compiled a list of 7 reasons that can cause a Joomla website to become super slow all of a sudden:

  1. Hacked website: When a website is hacked, it typically attempts to grab the contents of a malicious URL using PHP’s curl library or using PHP’s file_get_contents function (the latter can only work when the extremely unsafe allow_url_fopen PHP setting is enabled). This process is typically instantaneous, however, when the malicious URL gets blocked by the host or when the malicious website is hosted on a very slow (and cheap) host, the PHP function will wait for x seconds (where x is determined by the max_execution_time PHP setting) before timing out and serving the page. Obviously, the solution to this problem is to cleanup the website.
  2. Draconian firewall settings: Many hosts resort to SYN_FLOOD protection in order to minimize the impact of a DoS (Denial of Service) attack. While there is nothing wrong with that, some cheap/inexperienced hosts set the SYN_FLOOD value (at the firewall level) to a very low number, causing a very slow response time on the website. This is somehow ironic, because the firewall ends up causing the very same problem it was installed to protect the website from. The fix to this problem simply consists of asking the host to increase the SYN_FLOOD value.

  3. A switch to a slower hard drive: What most people do not know is that hard drives have a huge, huge impact on the performance of the website. In fact, when the website of a client of ours was moved from an SSD drive to an HDD drive, the performance degraded substantially, and the website suddenly became mostly unresponsive (at best it was super slow). Of course, the fix to the problem in this case was to switch back to an SSD.

  4. Disabled caching: There are many Joomla administrators who give super user access to people who don’t deserve it – typically, this is done because these people outrank the Joomla administrators. The problem with this is that some of these people are a bit technical, and, even though the word bit is the essence of programming, it becomes a huge curse when combined with the word technical or knowledge. Essentially, these super users will not only have enough rope to hang themselves, but also the whole website. For example, when a problem happens on the site, and they read that fixing the problem consists of clearing the cache, they go ahead and disable it altogether thinking that they fixed the problem forever. Shortly afterwards, the website becomes very slow or crashes, with nobody having the slightest idea of what might have caused the issue. A super quick fix to this problem is to re-enable caching on the site.

    Another problem causing this issue is the switch from one caching mechanism to another, for example, after switching a client site from File to Memcache caching, we experienced an increase in response time (although it wasn’t substantial) and we reverted back quickly.

  5. A Joomla plugin/extension: Some time ago, we investigated a Joomla website that was taking exactly 10 seconds to respond. At first, we thought it was hacked, but eventually, it turned out that there was a plugin called pingomatic which was using PHP’s curl to grab the contents of a remote URL that was no longer accessible, and so the request was timing out. Disabling the pingomatic plugin addressed the problem.

  6. High server load on shared hosting: If your website is on a shared host (or even a VPS), then keep in mind that other websites running on the same server can affect the performance of your website. If a website on the same server is resource hungry, and if not enough monitoring/control is done by the host, then this will become your problem and your website will become very slow. An ideal solution to this problem is to move to a dedicated server, a not-so-ideal solution is to tell your host about it and ask them if they can move the offending website elsewhere.

  7. Networking issues: In some instances, we have concluded that high latency was caused by networking issues/DNS issues. Naturally, fixing issues with routing/networking is outside the scope of developers, and is 100% the responsibility of the host. So, check with your host if you think that the sudden slowness on your Joomla website is caused by a networking issue.

We hope that you found this post informative and light, and that it helped you get to the bottom of the sudden performance issue on your Joomla website. As always, we are always here for you if you need professional help. Just contact us and we’ll ensure that the problem on your Joomla website is solved swiftly (well, as fast as we can) and affordably.

On the Joomla “onContentBeforeSave” and “onContentAfterSave” Events

If you’re into Joomla plugin development, then you are most likely familiar with the onContentBeforeSave and the onContentAfterSave events. The first event is triggered just before someone saves any Joomla content item, and the second event is triggered (you’ve guessed it) just after someone saves a Joomla content item.

The onContentBeforeSave event is typically used to alter the data that is about to be saved to the database, or to do something just before the data is saved.

The onContentAfterSave event is typically used to do something after the data is saved, for example, redirect to a specific page, or perform one or more database activities.

Now, if you read the first paragraph very carefully, you will understand that there is a problem. Since these events are run when any content item is saved, then this means that they can run at the wrong time. For example, if you have code that should be run when an article is saved, then this code will also run when you save other content items (such as categories or content items of non-core extensions)… In the absolute majority of cases, this is not a desirable behavior. So, in order to avoid this problem, developers add a condition in the beginning of the event to tell it which context it should run in. For example, if a developer wants to have the onContentBeforeSave run only when an article is saved, then he adds the following line to the beginning of the onContentBeforeSave function:

if ($context !== 'com_content.form') return true;

The above line ensures that the event runs only when an article is being saved.

What the absolute majority of Joomla developers don’t know is that there is another, more elegant solution to the problem, and it is by assigning different names to the above events based on the content type. The Joomla core already takes advantage of this hidden functionality: let us take a look at the model of the plugin extension, which is the file plugin.php which is located under the administrator/components/com_plugins/models. In particular, let us look at the constructor of that model:

public function __construct($config = array())
{
	$config = array_merge(
		array(
			'event_after_save'  => 'onExtensionAfterSave',
			'event_before_save' => 'onExtensionBeforeSave',
			'events_map'        => array(
				'save' => 'extension'
			)
		), $config
	);

	parent::__construct($config);
}

You can see that the event_before_save and the event_after_save configuration parameters were set to onExtensionBeforeSave and onExtensionAfterSave respectively. By default, these 2 configuration parameters are set to onContentBeforeSave and onContentAfterSave (this default is set in the admin.php file which is located under the libraries/legacy/model folder). So, when a Joomla plugin is saved, it triggers the onExtensionBeforeSave and onExtensionAfterSave events instead of the onContentBeforeSave and onContentAfterSave events. What does this mean? Well, it means that you can avoid triggering the default before save and the after save events by explicitly setting the values of the event_before_save and the event_after_save configuration parameters in your extension’s model.

We haven’t really discovered fire in this post, but we have demonstrated one best practice in the implementation of Joomla extensions, a best practice that we believe can benefit some Joomla developers out there by saving them some headache.

Now if you, our dear reader, need help with your Joomla events or with the development of Joomla extensions, then please contact us. We are experts in Joomla coding, our work is super clean, and our fees are really affordable!

A Simple Joomla Database Optimization Tip for a Huge Performance Gain

We love optimization work, there’s always something new to discover, and there’s always something new to learn. Our love for optimization led us to discover a gem, a little optimization gem that literally halved the load of a server powering an extremely large and high traffic Joomla website. As usual, we are always thrilled to share our discoveries with our readers, so here goes…

While investigating load issues on a huge Joomla website that we manage and that we previously optimized, we noticed that the following query was taking a relatively long time:

SELECT a.id, a.title, a.alias, a.catid, a.created, a.created_by, a.created_by_alias, a.created as publish_up,a.publish_down,a.state AS state FROM #__content AS a WHERE a.state = 1 AND (a.catid = 55 OR a.catid IN ('300', '301', '302', '303')) AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2017-09-07 18:09:39') AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2017-09-07 18:09:39') ORDER BY a.publish_up DESC, a.created LIMIT 20

The above query is a modification and an optimization of the core Joomla query that is used to generate the list of articles for pages/modules. The modification consists of displaying articles from multiple categories (instead of a specific category), while the optimization consists of an oldie but goldie optimization process that we first implemented on Joomla 2.5, and that is still valid until today (the website in question is Joomla 3.7.5). Having said that, the heart of the query is still representative of the Joomla core query, which means that any problem in the above query is also a problem in an unmodified and an unoptimized core Joomla article selection query.

In order to learn more about the issue, we went to phpMyAdmin, and we added an EXPLAIN directive to the beginning of the above query, and then we executed it. The query executed was as follows:

EXPLAIN SELECT a.id, a.title, a.alias, a.catid, a.created, a.created_by, a.created_by_alias, a.created as publish_up,a.publish_down,a.state AS state FROM #__content AS a WHERE a.state = 1 AND (a.catid = 55 OR a.catid IN ('300', '301', '302', '303')) AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2017-09-07 18:09:39') AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2017-09-07 18:09:39') ORDER BY a.publish_up DESC, a.created LIMIT 20

Here’s what we saw:

Explain Query Result

Figure 1: Joomla’s article selection query explained using the EXPLAIN tool

If you look at the key in the above image, you will notice that the MySQL engine chose idx_state (which is the state index) for the key, which is far from ideal. The key should be a combination of all the condition fields and all the order fields, which means that in our query above, the key should consist of the following fields: state, catid, publish_up, publish_down, and created. So, we created an index called idx_articles which consists of all those fields using the following query:

ALTER TABLE `#__content` ADD INDEX `idx_articles` ( `state`, `catid`, `created`, `publish_up`, `publish_down`);

After creating the above index, we re-issued the EXPLAIN query and here’s what we saw:

EXPLAIN Query Result After Indexing

Figure 2: Joomla’s article selection query explained using the EXPLAIN tool after adding the “idx_articles” index

As you can see in the above image, the MySQL database is now choosing the right index that is fully optimized for the article selection query. Additionally, the number of rows examined has decreased considerably.

So, how did this simple optimization affect the server load?

Marvelously, we can safely say! The load was more than halved. It was at 2.2, and it became 0.8, yes, it was that good! If you think it’s too good to be true, then try it and see for yourself (and let us know)! While we don’t guarantee that you will see the same extraordinary result that we saw (especially if you are using extensive caching on your Joomla website), we have the feeling that you will thank us for this.

Now, if you need help with creating the index or if you didn’t see any noticeable gain after creating the index, then please contact us. We are experts in optimizing Joomla websites, and our fees are super reasonable!

How We Integrated GetResponse with RSForm Pro on a Joomla Website

A client of ours wanted to integrate webinar purchasing on her Joomla website. She had a GetResponse account where she creates webinars, and she wanted to give her clients the ability to purchase her GetResponse webinars through her website.

Unfortunately, GetResponse does not technically allow either directly or indirectly the purchase of webinars. So, in order to have someone join a webinar, the person with access to the GetResponse account must manually create the contact (if it wasn’t created previously) and then add the contact to the webinar. Of course, this can only be done after charging the contact for the webinar, which was done manually over the phone. Obviously, we needed to automate all that, and so we leveraged the GetResponse API in order to do it.

Our vision was to have the contact visit a form, that (among other fields) displays a dropdown of all the GetResponse webinars, and then, have a payment method (on the same form) that the contact can use to pay for the webinar. Once payment is made and is successful, then the contact is added automatically to the webinar using the API.

So, the first thing that we did was to generate an API key on the GetResponse website (naturally, you will need to have a valid login information in order to do that).

We then downloaded the GetResponse API Client Library from here, and we modified it to accommodate our needs (we mainly added a few functions). The modified client library can be downloaded here. Once downloaded, we extracted the API file to the api folder (a folder which we manually created under the main directory of the Joomla website).

The next step was to download and install RSForm Pro onto the client’s website, and then download and install the RSForm Pro PayPal Plugin. Once that was done, we started creating the form. The form had the following fields: Name, Email, Webinar, rsfp_product, Payment Type, Total to be paid, PayPal, and Pay for Webinar. Here’s a quick explanation of all the fields:

  • The Name and the Email field are self-explanatory. They are both of type Textbox.
  • The Webinar was of type Dropdown and it had the following code in the Items field:

    if (!function_exists('getResponseWebinars')){
    	function getResponseWebinars(){
    		require_once(JPATH_ROOT.'/api/getresponse.php');
    		$objGetResponse = new GetResponse('your_getresponse_api_key');
    		$objWebinars = $objGetResponse->getWebinars();
    		$arrResult = array('|Select Webinar');
    		foreach ($objWebinars as $webinar){
    			$arrResult[] = $webinar->webinarId.'__'.$webinar->campaigns[0]->campaignId.'__'.$webinar->name.'|'.$webinar->name;
    		}
    		return $arrResult;
    	}
    }
    
    $ignoreThisValue='<code>';return getResponseWebinars();

    The above code dynamically populates the dropdown with all the webinars created in GetResponse. Note: If you are intrigued by the last line in the above code, then check this post for an explanation.

  • The rsfp_product field was of type Single Product and it was used to set the price of the webinar (note: all the webinars had the same price).

  • The Choose Payment field was of type Choose Payment, and it contained an automatically generated list of payment methods. In our case, the only payment method was PayPal.

  • The PayPal field was of type PayPal, and it was necessary to have PayPal as a payment method. Note that the PayPal payment settings are defined in RSForm Pro‘s configuration page (which can be accessed by going to Components -> RSForm! Pro -> Configuration).

  • The Pay for Webinar button was just a submit button.

Once we created the form above, we viewed it (without submitting any data) and it was indeed displaying the webinars in the dropdown.

Now, what remained was to perform the proper actions when a payment is successful. So, we created a plugin called RSFP Payment Success, which implements the rsfp_afterConfirmPayment event, which is triggered when a payment is successful. We added the following code in the rsfp_afterConfirmPayment function:

//get submission information
$db = JFactory::getDbo();
$sql = "SELECT FieldName, FieldValue FROM #__rsform_submission_values WHERE SubmissionId='".$SubmissionId."'";
$db->setQuery($sql);
$arrResult = $db->loadAssocList();
$arrFinalResult = array();
for ($i = 0; $i < count($arrResult); $i++){
	$arrFinalResult[$arrResult[$i]['FieldName']] = $arrResult[$i]['FieldValue'];
}

//now use the API to add the contact and then tag it
require_once(JPATH_ROOT.'/api/getresponse.php');
$objGetResponse = new GetResponse('your_getresponse_api_key');

$strName = $arrFinalResult['Name'];
$strEmail = $arrFinalResult['Email'];
$strWebinar = $arrFinalResult['Webinar'];
$arrWebinar = explode('__', $strWebinar);
$strWebinarID = $arrWebinar[0];
$strCampaignID = $arrWebinar[1];
$strWebinarTitle = preg_replace("/[^A-Za-z0-9]/", '', $arrWebinar[2]);

// Add the contact and associate it with a campaign
$result = $objGetResponse->addContact(array('email'=>$strEmail, 'name'=>$strName, 'campaign'=>array('campaignId'=>$strCampaignID)));

//Add the webinar name as a tag if it doesn't exist
$webinarGetResponseID = $objGetResponse->getOrCreateTagByName($strWebinarTitle);
$resultUpdateContact = $objGetResponse->updateContact($contact_id, array('tags'=>array('tagId'=>$webinarGetResponseID)));

The above code creates the contact if it doesn't exist, then it creates a tag generated from the title of the webinar (also if it doesn't exist), and then tags the contact with the webinar tag. Unfortunately, GetResponse doesn't have any means to add a contact directly to a webinar through the API, hence the workaround with using tags (this means that a slight manual work is required by our client in order to add all contacts with a certain tag to a certain webinar).

Have we had any hiccups with the integration?

Yes - we did. The first major hiccup is the one that we just mentioned, and it is the fact that GetResponse does not have an API method to add a contact to a webinar. We overcame this problem by resorting to tags. It wasn't the most elegant solution, but it was sufficient to the client.

The other major hiccup that we had, is that when a contact is first created in GetResponse, he must verify his account by clicking on a link in a welcome email. If he doesn't do that, then a contact ID will not be generated, and the tagging won't work. This problem, however, was somehow erratic, but, in order to avoid it altogether, we added a row in a table called #__cron_tag_contacts containing the email of the contact and the tag ID when a payment was successful. We then created a small file called tag_contacts.php, which was run by a cron every 15 minutes, and that checked the table for any contacts that were properly verified (e.g. they had an ID in GetResponse). Once a contact gets verified, the script tags it and then deletes its associated row from the table.

We hope that you found this post useful, and that it helped you with your integration with GetResponse. We have tried to provide as much information and guidance as we can about the subject, but, we reckon that the complexity with this integration is a bit high and that some readers may need help with the implementation. If you're one of those readers, then fear not, we are here for you. Just contact us and we'll implement this for you quickly, efficiently, and affordably!

Joomla Security Tip: Regularly Check the Physical Cron Files on Your Server

Most administrators check the cron jobs that they have on their server by just logging into their cPanel and clicking on the Cron Jobs link under the Advanced section.

The more advanced administrators check their cron jobs by typing the following command in the shell:

crontab -e

So, in the absolute majority of cases, an administrator does not directly open a physical file to check the cron jobs, he checks them through an interface whether through cPanel or the Linux crontab (or a different interface altogether). Now, the question is, where does the cron information come from? In other words, where is it stored?

Well, on a CentOS server (which is the typical server used for a WHM environment), cron jobs are stored in files under the directory /var/spool/cron. Each file is named after a user (each user is typically the username of a cPanel account), and each file contains the cron jobs for that particular user.

Now, you might be wondering, is there any point to this? Why should anyone check the actual physical file instead of using an interface to view the cron jobs? Well, because, in some cases, some information is not displayed in the interface, and, in most of these cases, this information is malicious. Let us give you an example…

Yesterday morning we were hired to add a cron job to a Joomla website, and, for some reason, we were having problems creating this cron job using the cPanel interface and using the crontab -e command, so, we went straight to editing the physical file containing the cron. Here’s what we did:

  • We ssh’d to the server as root.
  • We opened the file cpanel-user which is located under the /var/spool/cron folder (you will need to change cpanel-user to your cPanel username) for editing the following way:

    vi /var/spool/cron/cpanel-user

  • We saw this code:

    SHELL="/bin/bash"
    MAILTO=""
    DOWNLOAD_URL="http://[malicious-domain].com/u/w.gz"
    LOCAL_FILE_PATH="/home/[cpanel-user]/public_html/libraries/joomla/client/client.php"
    TMP_DIR="/var/tmp"

As you can see in the above code, only the first line is benign, the remaining lines are malicious. On the bright side, however, the code had no effect because it was missing the actual cron job. Its seems that the Joomla website we were working on had a cron job hack identical to the one we described here (a malicious file was being downloaded from DOWNLOAD_URL, and was being extracted to the client.php core Joomla file), and whoever fixed it didn’t fully cleanup the cron file (he just removed the cron job from the interface). So, we cleaned up the file and, after doing that, we were able to add the cron job through the interface.

So, it’s always a good idea to regularly check the contents of the files located under the /var/spool/cron folder. If you find anything suspicious and you are not sure what to do, then please contact us. We are experts in Joomla security, our work is quick, and our fees are always affordable!

How to Change the “Last-Modified” HTTP Header in Joomla + Free Plugin

A regular client of ours came to us yesterday evening with an interesting task: she wanted to set the Last-Modified HTTP header on an article page to the actual modified date of an article. We thought it was an easy task, until we started working on it.

You see, by default, Joomla sets the Last-Modified HTTP header to the current date, unless of course, it is using caching, in that case it sets the Last-Modified HTTP header to the cache time of the article. Logically, the Last-Modified date should be the last modification date of the article, but, in most scenarios, nobody cares about this. However, our client was using a tool which relied on the Last-Modified HTTP header in its data collection and display.

So, we dug in the code to find where the Last-Modified HTTP header was set, and we quickly (well, not so quickly) discovered that it was in the file web.php which is located under the libraries/joomla/application folder. In fact, the whole thing was done in this line:

$this->setHeader('Last-Modified', gmdate('D, d M Y H:i:s') . ' GMT', true);

As you can see in the above line, the Last-Modified HTTP header is set to the current date and time; it completely ignores the modification time of any item the user is currently viewing. If we want to change the Last-Modified value to the actual modification time of the article, we’ll have to replace the above line with the following code:

$currentView =JFactory::getApplication()->input->get('view');
$currentOption =JFactory::getApplication()->input->get('option');
if ($currentView == 'article' && $currentOption == 'option'){
	$currentArticleID = JFactory::getApplication()->input->get('id');
	$sql = "SELECT `modified` FROM `#__content WHERE `id`='$currentArticleID'";
	$db->setQuery($sql);
	$strDateModified = $db->loadResult();
	$objDateModified = new JDate($strDateModified);
	$strDateModified = $objDateModified->format('D, d M Y H:i:s').' GMT';
	$this->setHeader('Last-Modified', $strDateModified, true);
}
else
	$this->setHeader('Last-Modified', gmdate('D, d M Y H:i:s') . ' GMT', true);

The above code will work, however, there is one little problem with it: it is a core modification, and that’s why, at itoctopus, we have devised a better method to do this job, we have developed a plugin, and you can download it here! All you need to do is to just download the plugin, install it, enable it, and that’s it! Your article pages will display the actual last modification date of the article, and not the current date.

Now, here are some FAQs about the plugin:

  • Is it really free?

    Yes – it is free. You can use it anywhere and for anything. You can modify it and do anything with it. An attribution to itoctopus would be nice, but not necessary.

  • Is it supported?

    No – we do not offer free support for any of our extensions, since all of our extensions are free. If you would like support for any of our extensions then please keep in mind that our fees apply.

  • Does the plugin only work with Joomla articles? Can it work with K2 items?

    As it is, the plugin only works with Joomla articles, but, it can easily modified to accommodate other types of content items, such as K2 items. If you need help in this area, then please contact us.

Finally, we would like to stress the point that the plugin is provided as is, without any explicit or implicit warranty/responsibility. It may or may not do what we claim it does, and it may or may not destroy your website or our entire solar system if used, we don’t know, so use at your own risk! Again, we don’t assume a shred of responsibility for this plugin, so use at your own risk!

Oh, and by the way, we have a quick note to the Joomla team, the onAfterRespond can never ever be triggered. Please check the code in the respond function; it took us ages to discover that this event does not work!

How to Display Articles from Multiple Categories on a Category Blog Page

Note: The solution in this post is a modification to a core file. Proceed at your own risk, and keep in mind that a Joomla update may wipe out your changes.

We got a request from a client of ours to display articles from multiple categories on an existing category blog page. As a Joomla administrator, you probably know that a basic installation of Joomla can’t do this. In fact, this is the main reason (in most cases) why a Joomla website uses K2 for content management instead of Joomla’s core.

Luckily, we have done this before for another client, so we processed the request swiftly. Here’s what we did:

  • We logged in to the Joomla backend and we opened the existing menu item pointing to the category blog page.
  • We checked the string in the Link field, which was this one:

    index.php?option=com_content&view=category&layout=blog&id=50

  • We took note of the number “50” (which is the ID of the category) in the above string.

  • We then gathered the IDs of all the categories which the client wanted their articles displayed under the above category blog page. Here are the IDs: 77, 81, 107, 155.

  • We opened the file articles.php which is located under the components/com_content/models folder.

  • Just below the following line:

    $categoryEquals = 'a.catid ' . $type . (int) $categoryId;

    We added the following line:

    if ( (int) $categoryId == '50'){
    	$categoryEquals .= ' OR a.catid IN (\'71\', \'81\', \'107\', \'155\')';
    }

    Note: Of course, you will need to replace “50” with your category number. You will also need to replace “71, “81”, etc… with the IDs of the additional categories you want to display articles from.

  • That’s it! After doing the above, the category blog menu item displayed articles from all the above categories.

Note that you will need to create hidden menu items of type Category Blog, each pointing to a category in the above list to ensure that SEF links are properly generated for your articles.

But, what about using subcategories to display articles from multiple categories?

Well, you can do that, provided the multiple categories are immediate children of the main category (e.g. the category with ID “50” in the case of our client). If this is the case, then all you need to do is to open the menu item pointing to the main category, and then click on the Blog Layout tab, and finally set the value of Include Subcategories to “Yes”.

However, if you are stuck with dispersed categories that are not children of the main category, then the quickest solution would be to use our core modification above.

We hope that you enjoyed our simple post for displaying articles from multiple categories on a category blog. If you need help with the implementation, then just contact us. We always love having more clients, our work is super fast, our expertise is top notch, and our prices are super affordable!

“Invalid mime type detected.” Error When Trying to Upload a PDF File in Joomla’s Media Manager

Note: The second solution presented in this post consists of a core modification, which means that a future update can overwrite your changes. Always document your core modifications so that you can re-apply them after an update.

During the past weekend, we updated the website of one of our clients to Joomla 3.7.5 from Joomla 3.6.5 as we thought the former was stable enough. Yesterday, the client emailed us and told us that whenever they upload a PDF file through the media manager, they were seeing the following error…

Invalid mime type detected.

…and, of course, the upload was failing.

We investigated the issue and it turned out that there was a change in Joomla’s 3.7.x file checking process: In Joomla 3.6.5, a function called canUpload (in the media.php file which is located in the libraries/cms/helper folder) checks if the file uploaded is legal (e.g. it is allowed). If it is, then it allows the upload. In Joomla 3.7.x, the same function, canUpload, is triggered, but, in a default scenario, it also tries to guess the MIME type of the uploaded file using the function getMimeType. If it can’t (for any reason), then it returns false, and will display the above error.

Now the question is, what is a MIME type?

MIME is acronym for Multipurpose Internet Mail Extension, and it is just a string representing the type of a file and is typically used by the browser to know which type of software to launch to display a certain file. For example, if the MIME type of a file is application/pdf, then the browser will know that it needs to open the Acrobat Reader software to display the file. On the server end, the MIME type is often used to know the type of the file being uploaded, in order to decide whether to allow the upload or not.

However, the PHP function that is used to return the MIME type, which is mime_content_type, is an unstable function. In fact, this function was deprecated and then un-deprecated. There is, however, another method to return the MIME type, and it is by using the function finfo_open to return a fileinfo object out of a file and then using the function finfo_file to return the MIME type of the file. Now the problem with the last method is that it requires the PECL PHP library installed, which is not installed by default. The function getMimeType which is called by the function canUpload, first uses the first method to determine the MIME type of the uploaded file, and then it uses the second method. If it can’t determine the MIME type of the uploaded file using either method, then it returns false, leading to the Invalid mime type detected error.

So, what is the solution to the problem?

Since the MIME type check is done only when the Restrict Uploads setting in the Media Manager configuration is set to “Yes”, then a very quick solution would be to change that setting to “No” by logging in to the backend of the Joomla website, and then going to Content -> Media, and then clicking on Options on the top right, and then flipping the value of Restrict Uploads from “Yes” to “No”. Now, if someone hovers on Restrict Uploads, then he will see the following message:

Restrict uploads for lower than manager users to just images if Fileinfo or MIME Magic isn’t installed.

The problem is, however, is that the above message isn’t accurate. In fact, when Restrict Uploads is set to “Yes”, then it’ll be applied for all users, even super users, which is wrong. There is no check anywhere for the user type in the actual code. In order to fix this bug (which will also fix the original problem and will address any security implications resulting from setting the value of Restrict Uploads to “No”), you will need to do the following (note that if you change the Restrict Uploads value to “No” then you don’t need to do the below):

  • Open the file media.php which is located under the libraries/cms/helper folder.
  • Change the following line:

    if ($params->get('restrict_uploads', 1))

    to the following line:

    if ($params->get('restrict_uploads', 1) && !JFactory::getUser()->authorise('core.admin'))

  • Save the file and upload it back.

  • The problem should be solved!

Of course, there is another solution that consists of installing the proper PHP libraries, but this solution is dependent on the type of the hosting environment, so check with your host/system administrator about this (have them install the PECL PHP library or the MIME Magic library).

We hope the you found our post useful and that it did solve your problem. If it didn’t, then please contact us, we’ll help you address this problem for you in no time and for a very, very reasonable fee!

Joomla Performance Tip: Delete Old Entries in the “content_frontpage” Table

Unlike WordPress, Joomla has the unique ability that allows an administrator to label an article as a featured article, which means that the article will display on a page where the menu item is of type Featured Articles (typically, this page is the homepage). You might think, well, that’s not so special, but there is more to this: Joomla allows the administrator to have a different order for featured articles, and that order is independent from their order within their respective categories.

If you’re the skeptical type, then you are probably thinking: “Well, OK! But that’s not really much!” And it isn’t, except that the underlying database structure and the code gets signficantly messy because of this. You see, in order to support the above functionality, Joomla queries an additional table on a page with a Featured Articles menu item (which, again, is typically the homepage), which creates a huge load on the server when that table is of considerable size. The solution to this problem is to regularly trim the #__content_frontpage table and ensure that it doesn’t contain more than 100 entries. This can be done the following way (please backup your database before proceeding further; if you don’t want to then at least backup the #__content_frontpage table):

  • Login to phpMyAdmin.
  • Click on the #__content_frontpage table on the left pane (of course, you will need to replace #__ with the database alias of your Joomla website).

  • Click on the SQL tab on the top and type in the following query:

    DELETE FROM `#__content_frontpage` WHERE `ordering` > 100;

  • Click on the Go button on the bottom right.

  • That’s it!

Once you do the above, you will feel a noticeable gain particularly if your #__content_frontpage table is quite large.

We hope that you found this post useful. If you need help with the implementation, or if you need to implement the above in a cron, then please contact us. We are always happy to serve, our prices are super affordable, and our work is super clean.

Joomla Displays Altered Content on Some Pages (Hint: It Is Hacked)

A few days ago, a client of ours emailed us and told us that some of the pages on her company website were displaying content that was not theirs. The content was not technically malicious, but it consisted of ads, which had nothing to do with the client’s business. Clearly, the website was hacked.

Since only a few pages were affected, we had a hunch that it was a database hack, so we searched the database for the modified content, and, oddly enough, we didn’t find the modified content in the #__content table, but we found it in a table called #__finder_data. We’ve never seen this table before. To the untrained eye, the table seemed to be part of the Smart Search extension, but, as Joomla experts, we knew that it couldn’t be the case, as the Smart Search does not use any such table.

So we looked at the content of the table, and we noticed that most of the rows were benign and were just a copy of some rows in the #__content table. A few rows, however, had some altered (hacked) content.

We quickly (and wrongly) assumed that the #__finder_data table was used by a 3rd party extension for some caching reasons (although it seemed pretty useless, as it was just a copy of the #__content table), but we needed to know which extension it was used by. So, we did a grep on the filesystem which quickly revealed that the table was used in the file query.php which is located under the libraries/joomla/database folder. Yes, you’ve guessed it, the query.php is a core file, which means that the whole #__finder_data is likely part of the hack. So, we opened the query.php, and we searched for finder_data, and we found the following code:

if (!isset($GLOBALS['issebot'])) $GLOBALS['issebot']=$this->is_sebot();
if ( @$GLOBALS['issebot'] AND preg_match('/#__content(\s|$)/', $tables) AND in_array($db_px.'finder_data', $this->db->getTableList() ) ) {
    $tables=str_replace('content', 'finder_data', $tables);
}

The above code was located in the from function, and it checked if the traffic was coming from a bot (the method is_sebot was also added to the query.php file), and if it was, then it displayed content with matching ID from the #__finder_data table. Clearly, the method to check if the traffic was coming from a bot or not wasn’t exactly working properly, as the website was displaying the modified content for regular, non-bot traffic.

We quickly did the following:

  • We overwrote the hacked query.php file with a clean query.php file from a matching Joomla version (actually, we used our super fast method to clean Joomla sites and overwrote the whole core).
  • We deleted the #__finder_data table as it had nothing to do outside the hack.

Doing the above solved the problem, but, of course, it didn’t answer the question, how did this happen in the first place even though the site was using a very secure version of Joomla? We don’t know for sure, but we suspect the host. We’re not saying that the host did this, but what we are saying that the host that the client uses is notorious for bad security (the host has the word giant in its name, you figure the rest).

If your Joomla website displays (even slightly) different content than the real content, then likely your website is hacked. Try our simple cleanup instructions above and see if that fixes the problem. If it doesn’t, then please contact us. We will clean your website, we will secure your website, and we won’t charge you much.

“Error loading component: com_fields, Component not found.” Error After Updating Joomla to 3.7.x

One of the nice features in Joomla is that you can update the CMS from either the backend or from the filesystem. We typically update our clients’ websites from the backend, but, when we encounter the minor of issues, we update it using the filesystem. Here’s how:

  • We ssh to the server as root and we cd (change directory) to a temporary directory.
  • We download the latest Joomla install using wget.

  • We extract the install in the temporary directory and then we remove the installation folder (and the templates folder if the site uses a built-in Joomla template).

  • We zip everything back and we move it to the root directory of the Joomla site.

  • We extract everything there and then we use a recursive chown to change the ownership and the group of each of the files to the right one.

  • We login to the Joomla backend and then we go to Extensions -> Manage -> Database and then we click on the Fix button on the top left.

  • That’s it! Typically, the above process takes a few minutes, and we only do it if the current Joomla website is a few revisions behind.

Lately, however, we are having a minor hiccup with the above process (especially when updating from Joomla 3.6.5 to Joomla 3.7.x), as we are seeing the following warning when we go the Article Manager page (or any backend page having to do with the articles):

“Error loading component: com_fields, Component not found.”

The above warning is caused by the fact that the new article system in Joomla has been modified to use the custom fields (which has been introduced in 3.7.x), but the custom fields were not properly installed onto the website. The solution to this problem is to use the “Discover” functionality in the Joomla backend. Here’s how to do this:

Login to the Joomla backend and then go “Extensions” -> “Discover”, and then click on the “Discover” button on the top left. Once you do that, you should see the following Fields extensions that should have been installed on your Joomla website (but were not):

  • Content – Fields (Site Plugin)
  • Fields (Administrator Component)
  • Fields – Calendar (Site Plugin)
  • Fields – Checkboxes (Site Plugin)
  • Fields – Colour (Site Plugin)
  • Fields – Editor (Site Plugin)
  • Fields – Imagelist (Site Plugin)
  • Fields – Integer (Site Plugin)
  • Fields – List (Site Plugin)
  • Fields – Media (Site Plugin)
  • Fields – Radio (Site Plugin)
  • Fields – SQL (Site Plugin)
  • Fields – Text (Site Plugin)
  • Fields – Textarea (Site Plugin)
  • Fields – URL (Site Plugin)
  • Fields – User (Site Plugin)
  • Fields – Usergrouplist (Site Plugin)
  • System – Fields (Site Plugin)

Select them all by clicking on the top checkbox (the one next to “Name”). Once you have them all selected, click on “Install” button on the top left. When the install finishes (it should take a few seconds), you should see the following message Discover install successful. You can then verify that the problem is fixed by going to the Article Manager (you shouldn’t see the warning any more).

We hope that you found our little post useful and we hope it helped you solve your problem. If it didn’t, then please verify that your server doesn’t have any type of aggressive caching. If you still need help, then you can always contact us. We are eager for new clients, our current clients love us, and our fees are always right.

Lack of Full Time Testers is Joomla’s Biggest Problem

A few weeks ago, we discussed the main advantages that WordPress has over Joomla. One of these advantages is the fact that WordPress has full time developers and a huge testing team. Of course, for those of us working with Joomla, we know that this isn’t the case for our beloved CMS.

Joomla is supported by volunteer developers (who are highly dedicated) and by a volunteer testing team. Now any programmer worth his salt knows 3 things: 1) he cannot say that his code is “fully tested” when he’s the only one who tested it, 2) any code should never ever be made live without thorough testing, and 3) a simple line of code may unknowingly impact many areas of a product.

If you know how development on the Joomla core works, then you will know that the heart of the problem is the word “thorough” in the previous sentence, simply because the testing is anything but thorough. Let us explain the issue with an example of how a bug is handled:

  • Someone reports a bug to the Joomla development team.
  • A Joomla developer addresses the bug.

  • Other developers test the feature that the bug fix addressed.

  • The bug fix is packaged and prepared for the next Joomla release.

  • The Joomla release is tested shortly before its launch. The testing consists of installing the new Joomla version with some demo data onto someone’s PC and tinkering with that install.

Obviously, there are some serious flaws in the above process:

  • The bug fix may cause issues in areas other than the area addressed by the bug.
  • The testing done does not take into consideration the impact of the bug fix on the product as a whole.

  • The testing is oblivious to the concept of “3rd party extensions” in Joomla and is done on a pure Joomla website.

  • The testing completely ignores the impact of this simple bug fix on large datasets, because the demo data that is used for the testing consists of just a few articles.

We can give you a proof for each of the points mentioned above. For example, a few revisions ago, Joomla changed the ordering algorithm of articles. The change was meant to address a performance issue caused by the old article ordering algorithm. Some developers tested the change and it was made live in the next revision. Less than an hour after the release, many Joomla administrators flooded the developers with complaints about the new ordering algorithm, as it confused them by pushing newer articles to the very bottom of the articles’ list. Obviously, a consensus was reached shortly after and the change was reverted.

There are many, many other examples, but we don’t think it’s necessary to list them all to get our point across.

But what is the solution?

The solution is for Joomla to hire a full testing team, and to give testing a priority, and to test with real data (e.g. copy real large websites onto a dev environment and then test these websites there). Obviously, more money is needed to accomplish this, and so Joomla must be slightly less conservative when it comes to its tight techniques to raising money.

We hope you found this post useful. If you have any questions about it, then please leave a comment below. If you need help with your Joomla website, then please contact us. Our work is clean, our skills are solid, and our fees are affordable!

The Hacked “index.html” File on Joomla Sites

A new client called us yesterday afternoon and told us that his company website was displaying a weird page instead of his normal page (his company sells restaurant equipment). The weird page consisted of some gibberish in random languages. It was a no-brainer that the website was hacked.

Since the client was in a hurry, we decided to clean the website using our super quick method for cleaning Joomla websites, but, to our surprise, it didn’t work: the website still displayed the page with gibberish content.

We thought, it might be that the template chosen was hacked, so we switched to the Beez 3 standard Joomla template (that was definitely fixed by the code overwrite), but that didn’t work either.

Naturally, the next step was checking the .htaccess file, maybe there was a malicious rule somewhere, but even after deleting the .htaccess file the site was still hacked. We also checked for malicious .htaccess files in higher directories, and we found none.

Finally, we decided to add a PHP die(‘under maintenance’); statement to the beginning of the main index.php file and see how the website reacts. To our surprise, the website still displayed the hacked page.

There were 3 scenarios in our mind:

  1. The website was located in a different directory than the one we were working on.
  2. The website was located on another server than the one we were working on.

  3. The server hosting the site had some aggressive caching techniques.

While checking for the above possibilities, we noticed something peculiar: there was an index.html file lurking in the root directory of the Joomla website. We opened the file in our text editor, and, unsurprisingly, the file contained the hacked HTML code. Deleting the index.html file immediately solved the problem.

But how did the hacked index.html file got there in the first place?

We’re not exactly sure. The website was using the latest Joomla version (Joomla 3.7.3) and all the 3rd party extensions were up to date, so, most likely the issue was an undiscovered vulnerability in one of the installed extensions. The client elected not to proceed further with the investigation so we can never be sure.

So, how can one protect his website from a similar issue?

Addressing the result (and not the cause) can be done by adding some .htaccess rules to block access to the following files in the root directory of the Joomla website:

  • default.html
  • default.php
  • index.html

A simpler way to address the problem consists of adding the following line to the very beginning of the .htaccess file:

DirectoryIndex index.php index.html default.html default.php

The above line ensures that the index.php file has priority over all the other files and is always processed first by the web server.

We hope that you found this post useful. If you have some questions about it or if your website ran into the same issue and you want us to do some forensic work done in order to unveil the root cause of the problem, then please contact us. We are security experts in Joomla, our work is very clean, and our fees are extremely affordable.

The Main Advantages that WordPress Has over Joomla

WordPress, the most used CMS in the world, with around 9 times the market share of Joomla, is not better than Joomla. In fact, we think that Joomla is a much better product, and this is why:

  • Joomla uses MVC in its code, WordPress doesn’t.

    The term Spaghetti Code is a great fit for WordPress. Joomla has always adopted a clean Model-View-Control framework which is very easy to work with.

  • Joomla is way better structured than WordPress.

    In WordPress, everything is called “a plugin”, with the exception of templates, which are called “themes”. In Joomla, you have plugins, modules, and components.

  • Joomla is generally a more polished product.

    Compare a Joomla backend to a WordPress backend and you will know exactly what we mean.

But, even though we think that Joomla is better than WordPress, we cannot ignore the fact that Joomla lags behind when it comes to market share. We believe that this can be attributed to the following reasons:

  • Updating WordPress from almost any version to the most current version is a seamless operation.

    Case in point: A year ago we updated the itoctopus website (yes, guilty as charged, we use WordPress on the itoctopus website, mainly because our website is mostly a blog) from a very old version of WordPress to the most recent one, and it literally took us a few minutes to do so. Imagine trying to migrate a Joomla 1.5 website to Joomla 3.7.2, will you be able to do it in a few minutes? We can’t, and if you can, then please send over your CV right now!

  • SEF links are automatic with WordPress.

    In Joomla, you have to create menu items, and then point these menu items to articles/categories/etc… in order to get a working link. Sure, you have full control over your links in Joomla, but this extra process is what keeps many webmasters from adopting Joomla. Not many people grasp the idea of menu items. On the bright side, however, Joomla will, in the near future hopefully, have a seamless integrated SEF system.

  • WordPress is a dictatorship.

    WordPress was created by one person, and that one person has a final say in almost any decision involving WordPress, including technical and financial decisions. Essentially, WordPress represents the unwavering vision of just one person. Joomla is a totally different story. Joomla’s financial, structural, and technical decisions are taken by different people with different opinions and different agendas. Naturally, this means that many of these decisions are incoherent and incompatible with each other, and what makes things worse is that the Joomla decision makers are often replaced, further weakening the long term vision of the product.

  • WordPress is wealthy.

    WordPress employs many monetization techniques that generates a lot of money. Joomla, because of its complicated decision making process (and because of some people with hidden agendas), was never really monetized properly. In fact, the same people making a killing out of Joomla are the staunchest opponents to a real monetization of the Joomla brand.

  • WordPress has real – full time developers. Joomla doesn’t.

    Since WordPress is wealthy, then it can afford to hire real good quality developers to support its products. So far, Joomla doesn’t have full time paid developers, it only has volunteer developers. Even the lead Joomla developer is a volunteer developer. Of course, that it not a bad thing, but it is definitely not as good as when you have full time developers supporting the product, and a huge testing team testing the product.

  • The WordPress plugins directory is better maintained than Joomla’s JED

    The JED (Joomla Extensions Directory) is just non-comparable to the WordPress plugins directory, and this is because the JED is maintained by very few people. So, the approval, rejection, and the testing of an extension takes a long time and is often done just to “empty the plate”. This means that in many cases, no real testing is done, and this is especially the case for large extensions.

Due to the above factors, WordPress has a much larger community than Joomla, but, on the other hand, Joomla’s community is much more fanatic about Joomla (which can be a good thing and a bad thing). We know that Joomla is a better product in its core, but, we also know that without addressing the above advantages that WordPress has over Joomla, our beloved CMS will, at best, be in second place.

The above post reflects our opinion which is based on our experience with both products. If you have have something to say about this, then feel free to comment below. If you want to hire us to advise you on which CMS is best for you, then please contact us. Our fees are right, our work is honest, and we really know what we are talking about!

Joomla Performance Tip: Investigate Large Database Tables

One of the common bottlenecks on any website, and not just a Joomla website, is large database tables. Large tables, especially those with significant and constant CRUD (Create, Read, Update, and Delete) activities, can cause serious load issues on websites. This means that it’s always a good idea to check your Joomla websites for large tables and see if you can trim them.

For example, if you are using Smart Search, then you will notice that adding/editing Joomla (or K2) articles gets slower and slower with time, and this is caused by the exponential growth of the finder tables (check point #2 here for more information about the harm that is caused by Smart Search).

Another example is the ACYMailing tables, especially those that are used for storing stats. If you are an avid user of ACYMailing, then you probably know what we mean: a load spike when you are sending out emails (because of the new rows added to the ACYMailing stats table), and a load spike when people start opening their emails (all these database updates happening during a relatively short period of time).

Now, the question is, how can you check your database for large tables? Simple! You just go to phpMyAdmin, select the database powering your Joomla website, and then click on the Rows header. This will display the tables sorted by the number of rows descending (e.g. the tables with the most rows will display first). Once you know which tables have the most number of rows, you will be able to formulate a strategy in order to address that (of course, your options are very limited for some tables, such as the #__content table).

If you need help with those large tables, then all you need to do is to contact us. Our prices are right, our work is professional, and we always have a solution for any Joomla issue.

Varnish Caching and Invalid Tokens on Joomla Websites

A new client emailed us back on Saturday (today is Monday), and told us that he was seeing the famous (and dreaded) Invalid Token error whenever he tried to login to his Joomla website. He told us that the whole thing happened all of a sudden. He assured us that he didn’t install a new extension, he didn’t modify the settings of an extension, and he didn’t even add/update anything on his website in the past few weeks.

Of course, our first guess was that the website was hacked, so we did a thorough scan to the website, and it was as pure as the driven snow. Our second guess was a recently installed plugin, but there were none. So we disabled all the non-core plugins, and we still had the problem. Our third (and obvious) guess was Joomla caching, but there was no type of Joomla caching enabled.

So, we started debugging the problem by echoing messages in core files (just to narrow down the problem), and while doing that, we noticed something interesting, our changes were not reflecting immediately. Could it be? So, we did additional tests, and we were able to confirm that aggressive caching was being used by the host. So, we called the host and we were told that Varnish Caching was being used for that website, and so we asked them to disable it (they reluctantly agreed). Once Varnish Caching was disabled, everything worked perfectly, and the Invalid Token issue was resolved.

If you are having an Invalid Token error when you are trying to login to your website, then check our previous post about it. If it doesn’t work for you, then check with your host whether they are using server caching (such as Varnish Caching) for your website. If they do, then ask them to disable it. If they refuse to disable it, then maybe it’s time to move to a new host. If the problem has nothing to do with server caching, then all you need to do is to contact us. We are eager to help, our fees are super affordable, and we always find a solution.

How to Run a Joomla Administrator View from a Cron

Every few months, we get a request to execute a Joomla administrator view from a cron job. For example, a recent request consisted of running an administrator view from a cron job in order to grab data from other sites and save them into the Joomla website. Now, if you have dealt with cron jobs before, you will probably immediately answer that it’s possible, but (as always), there is a catch: How will you run the administrator view without logging in? If you can do that, then there is a major security exploit in Joomla that must be addressed immediately, but you can’t, at least not before reading this post!

So, how do we do run a Joomla administrator view from a cron job at itoctopus?

Well, we do it the following way:

  • We login to the backend of the Joomla website and then we create a super user called admincron with a random password.
  • We leverage the power of the defines.php file – that is also loaded by the index.php file located under the administrator folder to do the following:

    • Temporarily disable any security plugins (such as AdminTools, RS Firewall, etc…) by renaming the plugin’s folder to a different name.

    • Load the Joomla environment.

    • Login the user admincron to the Joomla backend.

    • Execute the view.

    • Halt further execution of the script.

  • We create the cron job to execute the administrator view.

Here’s the code that we add to the defines.php file:

<?php 
	$hash = $_GET['hash'];
	if ($hash == 'averystronghash'){

		// Optional: Disable any firewall plugin by renaming its folder to *_old
		rename('../plugins/system/firewallplugin', '../plugins/system/firewallyplugin_old');

		//Load the Joomla environment
		if (!defined('_JDEFINES'))
		{
			define('JPATH_BASE', __DIR__);
			require_once JPATH_BASE . '/includes/defines.php';
		}

		require_once JPATH_BASE . '/includes/framework.php';
		require_once JPATH_BASE . '/includes/helper.php';
		require_once JPATH_BASE . '/includes/toolbar.php';
		$app = JFactory::getApplication('administrator');

		// Login the "admincron" user to the Joomla backend
		$credentials = array();
		$credentials['username'] = 'admincron';
		$credentials['password'] = '[password]';
		$app->login($credentials);

		$app->execute();

		// Optional: Re-enable any firewall plugin that was disabled at the beginning of the script
		rename('../plugins/system/firewallplugin', '../plugins/system/firewallplugin');

		// Halt further execution of the script
		die();

	}

?>

Once you add the above defines.php file, you will be able to load any administrator view on your Joomla website without explicitly logging in. You will also be able to load any administrator view from the cron, by simply adding the following task to your cron job:

/usr/bin/wget "http://www.[yourjoomlawebsite].com/administrator/index.php?option=com_[yourcomponent]&view=[yourview]&hash=averystronghash"

Aren’t there any security implications for doing the above?

Not as long as you are using a very strong hash. Otherwise, you will risk allowing anyone who knows the link (in the cron task above) to login as super user to your Joomla website.

We hope you found this post informative and useful. If you have any questions about it, or if you need help with the implementation, then please contact us. We are always excited for working with new clients and our fees are very affordable!

“JHtmlBehavior::polyfill not found” Fatal Error on Joomla

A new client called us this morning and told us that the was seeing the following error on his website when he was visiting it:

An error has occurred. 500 JHtmlBehavior::polyfill not found

So, we visited his website in order to check what is going on, and, to our surprise, we didn’t see the error. We deleted the Joomla cache of the website and we asked the client to clear his browser cache and check the site again, but he still saw the problem. We starting suspecting that it was a propagation issue and that the client was seeing a different version of the site, but, after asking the client to ping his website, we realized that he was seeing the same website as us.

So, we dug into the Joomla code searching for the polyfill error – and we quickly discovered something very interesting in the file behavior.php which is located under the libraries/cms/html folder. It was these 3 lines:

// Include core and polyfill for browsers lower than IE 9.
static::core();
static::polyfill('event', 'lt IE 9');

Aha! It seems that the polyfill library was only being loaded if the person was using an IE browser that is less than Internet Explorer 9 (the client was using IE 8), but, why was it generating the error when Joomla tried to load it?

Further digging revealed that the polyfill library in Joomla 3.7.2 and in earlier versions does not support PHP 7 – it only supports PHP 5.5 and PHP 5.6, and since the client was using PHP 7, he was seeing an error.

So, what was the solution to the problem?

The solution to the problem was quite easy, all the client had to do was to use a different browser (he went with Google Chrome), and the problem was solved!

But what if the client didn’t have the option to switch to another browser?

Well, in that scenario, the client had to comment out the following line from the aforementioned behavior.php:

static::polyfill('event', 'lt IE 9');

Although we have to say, that commenting the above line may have other implications (we haven’t tested it to confirm) – so, as usual, proceed at your own risk.

We hope that you found our post helpful. If you are still seeing “JHtmlBehavior::polyfill not found” error on your website even after switching to another browser, then please contact us. We will always find a solution, our rates are super affordable, and our work is super clean!

Joomla Security Tip: Block Long URLs

At itoctopus, one of the things we are fascinated by is security – we are always researching new ways to improve the security of our managed websites. Our Joomla honeypot experiment, for example, was a huge success and we implemented it for our major clients. It also helped us better understand attack patterns, which, in turn, helped us develop the appropriate counter measures.

A common pattern in attacks on Joomla websites, as we noticed from our Joomla Honeypot Experiment, is long URLs. When a URL is abnormally long, then it’s almost a certainty that it’s an attack, and not a legitimate visit. An obvious counter measure would be to block long URLs from accessing the website. But how? And, how about false positives resulting from this aggressive security measure?

Well, the first thing that needs to be done is to check the Apache access logs, and generate a list of all the URLs sorted by their length. Here’s how (we are assuming that you are using WHM/CentOS as your hosting platform):

  • ssh to the server as root.
  • Change to the /usr/local/apache/domlogs by issuing the following command:

    cd /usr/local/apache/domlogs

  • Run the below code (substitute yourjoomlawebsite.com with your domain):

    awk '{print $7 }' /usr/local/apache/domlogs/yourjoomlawebsite.com | sort -nr | uniq -c | awk '{ print length($2) " " $2;  }' | sort -nr -k1 > urls-by-length.txt

    Note: The above code may take some time on high traffic websites, so please be patient.

  • Open the file urls-by-length.txt, and you will see a list of all the URLs accessed, reversely sorted by length. The length of each URL will be printed next to it.

  • Take note of the length of the longest legitimate URL that you have on your website (you will find it at the top of the list, not necessarily in the first row, as the first row may contain an attack URL). You will need it afterwards.

Now that you have the length of the longest legitimate URL on your website, you can create a simple condition in the defines.php file to block URLs that exceed the maximum legitimate length. Here’s how:

  • Open or create the file defines.php under the root directory of your Joomla website.
  • Add the following code to it:

    <?php
    $maximumAllowedCharacters = [number you got from running the awk command above + some padding];
    // Replace 'http' with 'https' if your website runs in SSL mode
    $currentURL = 'http://'.$_SERVER['HTTP_HOST'].$_SERVER['REQUEST_URI'];
    if (strlen($currentURL) > $maximumAllowedCharacters){
    	header('HTTP/1.0 403 Forbidden');
    	die('Unauthorized');
    }?>

  • That’s it! Now abnormally long URLs will be blocked from accessing your website.

Note that this can also be done at the ModSecurity level (if you have it installed on your server), and it is even more efficient to do it in ModSecurity, because multiple requests from the same IP can be blocked at the server level. Of course, ModSecurity is a huge realm, and developing a rule that enforces the above simple condition is not what anyone would consider a walk in the park.

But what about false positives?

There is a possibility that you will have false positives when implementing the above on your website, and that’s why it’s always a good idea to monitor your logs (especially 403 errors) in order to address any false positive.

Of course, there is much room for improvement in the defines.php code above. So if you are interested in enhancing it so that it works seamlessly on your website, then please contact us. We are experts in Joomla security, we are hard workers, and our fees are not scary!

How to Remove the PHP End of Life Warning in Joomla 3

We are getting calls from Joomla administrators telling us that they are seeing a bizarre warning in the backend of their Joomla sites after updating their sites to 3.7.x. Here is a sample warning:

Your PHP version, 5.6.30, is only receiving security fixes at this time from the PHP project. This means your PHP version will soon no longer be supported. We recommend planning to upgrade to a newer PHP version before it reaches end of support on 2018-12-31. Joomla will be faster and more secure if you upgrade to a newer PHP version (PHP 7.x is recommended). Please contact your host for upgrade instructions.

Most of the Joomla administrators contacting us are using the words bizarre or weird because they just don’t understand why they are seeing a warning for something that will happen at the end of 2018 (today is May 30th, 2017). Naturally, they ask us for an explanation, and, as always, we oblige:

Joomla versions 3.7.0 and higher have a quickicon plugin that gets the version of the PHP instance running on the server that the Joomla website is running on and then compares it to a hardcoded database of PHP versions, with their Security Fixes Only and End of Life (or EOL) dates. If the current date (e.g. today) is higher the Security Fixes Only date, but lower then the EOL date for the PHP instance, then this means that the PHP version is longer in Active Support, and is in the Security Fixes Only phase. When this happens, Joomla displays something like the above warning.

If the current date is even higher than the Security Fixes Only date for the PHP instance, then this means that the PHP version used has reached its End of Life, and Joomla will display something like the below warning:

We have detected that your server is using PHP 5.5.38 which is obsolete and no longer receives official security updates by its developers. The Joomla! Project recommends upgrading your site to PHP 5.6 or later which will receive security updates at least until 2018-12-31. Please ask your host to make PHP 5.6 or a later version the default version for your site. If your host is already PHP 5.6 ready please enable PHP 5.6 on your site’s root and ‘administrator’ directories – typically you can do this yourself through a tool in your hosting control panel, but it’s best to ask your host if you are unsure.

As you can see in the second scenario, the Joomla website will display a warning asking the administrator to update to a PHP version that will also display a warning – this is because the algorithm works as follows:

  • If the Joomla website is using a PHP version that is still supported, then do not display any message.
  • If the Joomla website is using a PHP version that is only supported through security patches, then display the first message.

  • If the Joomla website is using a PHP version that is no longer supported, then search for the first PHP version that is supported (even with security patches), and display the second message.

Obviously, the algorithm of the last step should be modified to return the PHP version that is still in Active Support.

In any case, most, if not all Joomla administrators feel uncomfortable when they see any of the above messages, and many of them do not have the time to update the PHP version on their server. Luckily, there is a way to get rid of these annoying messages. Here’s how:

  • Login to the backend of your Joomla website.
  • Go to on Extensions -> Plugins.

  • Search for the plugin titled Quick Icon – PHP Version Check and disable it by clicking on the green checkmark next to it.

  • That’s it!

As you can see, it is easy to get rid of the warning, but Joomla’s advice is sound, and you should update to a supported PHP version whenever is possible for the sake of both the security and the stability of your website.

Now if you’re wondering which PHP versions are the ones that do not display these messages, then they are, at the time of writing this post, all the PHP 7.x versions (including PHP 7.0).

We hope that you found this post useful. If you want to update your PHP version to the latest one but you are afraid that this may break something on your website, then please contact us. We’ll do the work for you, we won’t charge you much, and you will sleep better at night!

Joomla’s RSForm Pro PayPal Plugin Not Changing the Payment Status from “Pending” to “Accepted”

RSForm Pro is one of the most powerful extensions out there. You can create any type of form with it, and associate any action you can think of when the form is submitted, whether through plugins or through embedded PHP code. This makes RSForm ideal for integrating Joomla with payment gateways (such as Authorize, PayPal, and Stripe) and marketing tools (such as HubSpot, Marketo, Salesforce, and the likes).

But, hiccups are bound to happen from time to time, and we experienced a small hiccup with our last RSForm Pro integration. Here’s what happened…

A client of ours commissioned us to create a “gift card” page on their website, where people can purchase gift cards that are used towards subscriptions (the subscription system used was Akeeba Subscriptions – the gift cards are actually coupons in Akeeba Subscriptions). We used RSForm Pro for that, and we added 2 payment methods: PayPal and Stripe. The gift card process went as follows:

  • The client visits the page containing the form, which asks the client about his name, his email, the type of gift card that he wants to purchase ($50 gift card or $100 gift card), and his payment method (credit card or PayPal).
  • Based on the client’s choice of payment method, the system will either display a popup asking for the client’s credit card details or will redirect the client to PayPal.

  • When the payment is processed, the Stripe plugin and the PayPal plugin will trigger the rsfp_afterConfirmPayment event, which is implemented in the RSForm Payment Success plugin (which we developed). The RSForm Payment Success plugin will generate the coupon code on the Akeeba Subscriptions system, and will send the coupon code to the client.

  • The client is then redirected to a page displaying the coupon code (essentially a page containing the same information that was sent to the client in the email).

There was, however, two small problems in the process above: the payment status was not changed to “Accepted” (from “Pending”) and the rsfp_afterConfirmPayment event was not being triggered when the payment was completed through PayPal, which was odd, considering that, unlike the Stripe plugin, this plugin was not a 3rd party plugin.

Naturally, the first thing that we checked was whether the notify_url was correct, and it was. Luckily, while checking that, we noticed that the PayPal RSForm plugin was logging everything to a file named rsformpro_paypal_log.php in the logs folder, and so we immediately checked that file, which contained the following (very helpful) information:

2017-05-23 23:28:26 : IPN received from PayPal
2017-05-23 23:28:26 : Connecting to https://www.paypal.com/cgi-bin/webscr to verify if PayPal response is valid.
2017-05-23 23:28:26 : PayPal reported a valid transaction.
2017-05-23 23:28:26 : Check the order's amount paid: 50.00 - 50.00. Payment amount is correct.
2017-05-23 23:28:26 : Validation failed -> The email address is not correct - [email-address]

As you can see, the last line is saying that the email address is incorrect, but it was. So, we searched for the code that was throwing this error, and it was the following code in the file rsfppaypal.php which is located under the plugins/system/rsfppaypal folder:

if (RSFormProHelper::getConfig('paypal.email') !== $validation_fields['receiver_email'])
{
	$array['error']  = true;
	$array['reason'] = sprintf('The email address is not correct - %s', $validation_fields['receiver_email']);

	return $array;
}

At first glance, nothing seemed to be wrong with the above code, but, on closer examination, we discovered a subtle bug: what if the email set in the configuration is not exactly the same as the email sent back from PayPal, what if, for example, the email set in the RSForm configuration is something like myemail@myjoomlatestwebsite.com and the PayPal email is something like MyEmail@myjoomlatestwebsite.com – in that case, the condition (in the first line) will return true, and an error will be returned by the PayPal plugin (which will halt the completion of the PayPal transaction from RSForm’s end). So, we changed the above code to the following…

if (strtolower(RSFormProHelper::getConfig('paypal.email')) !== strtolower($validation_fields['receiver_email']))
{
	$array['error']  = true;
	$array['reason'] = sprintf('The email address is not correct - %s', $validation_fields['receiver_email']);

	return $array;
}

…and then we tested the form again, and this time, it worked. The payment status was changed to “Accepted” and the rsfp_afterConfirmPayment event was finally triggered on successful PayPal payments. Phew!

Now if you, our dear reader, have problems with PayPal transactions still set to “Pending” (instead of “Accepted”) in your RSForm Pro form submissions even though these transactions were successful, then it might be that the email in the configuration settings of RSForm Pro does not exactly match that in PayPal. You can either ensure that both emails are exactly the same (e.g. they have the same casing), or modify the RSForm Pro PayPal as described above. If you need help with the implementation, or if that doesn’t solve your problem, then please contact us. Our prices are super affordable, our work is super clean, and our turnover is super quick!

Saving Articles Timing Out After Updating to Joomla 3.7? Read This!

A client with a huge Arabic news website called us on April 26th (the day Joomla 3.7 was released) and told us that he’s not able to save articles after updating to Joomla 3.7. He told us that article saving was timing out, and urged us to take a look.

As usual, we obliged. We started our investigation by checking whether the issue is caused by one of the 5 reasons we listed in an earlier post. Unfortunately, it was none of them.

We then went to testing the website, and we noticed something odd: the problem was only happening when there is a lot of Arabic content in the article, because when we reduced the article’s content to a few Arabic sentences, the saving process went smoothly. When the article’s content was left as it was, the saving process was timing out. This gave us an idea:

  • We reduced the PHP maximum execution time to 30 seconds by creating a local .user.ini file at the root directory of the Joomla website and adding the following to it:

    max_execution_time = 30;

  • We changed Error Reporting to Maximum under System -> Global Configuration -> Server.

  • We tried saving the article (using the original, long Arabic content).

  • We saw the following error after 30 seconds:

    Fatal error: Maximum execution time of 30 seconds exceeded in /home/[cpanel-user]/public_html/libraries/vendor/joomla/string/src/phputf8/mbstring/core.php on line 41

Aha! We had a lead! We didn’t know what the problem was exactly but we had a lead. So, we opened the file core.php which is located under the libraries/vendor/joomla/string/src/phputf8/mbstring and we checked the problematic line, and it was this line…

return mb_strpos($str, $search, $offset);

which was located in the following function:

function utf8_strpos($str, $search, $offset = FALSE){
    if ( $offset === FALSE ) {
        return mb_strpos($str, $search);
    } else {
        return mb_strpos($str, $search, $offset);
    }
}

Hmmm! We felt, for some reason, that there was an infinite loop going on, and so we added a couple of lines to write the following variables $str, $search, and $offset to a file (we did it just before the problematic line), and, then we tried to add the article. Unsurprisingly, the file was several megabytes large after our test (mostly the same content being repeated over and over), so there was definitely an infinite loop going on.

Now, if you look at the above function, you will notice that it’s not recursive and doesn’t have any kind of loop, so, it must be the function calling it that is running into an infinite loop. Luckily, PHP has a very sweet method to know which function is the caller function, which is this line:

echo('Calling function: '.debug_backtrace()[1]['function']);

So, we used the above line to know which function was the caller function, and it was the function strpos which is located under the libraries/vendor/joomla/string/src folder. The following is the implementation of the strpos function:

public static function strpos($str, $search, $offset = false)
{
	if ($offset === false)
	{
		return utf8_strpos($str, $search);
	}

	return utf8_strpos($str, $search, $offset);
}

As you can see, the strpos function is not recursive, which means that the problem is caused by whichever function is calling it (or whichever function is calling the function calling it, or whichever function is calling the function that is calling it, OK – you get the idea!), ultimately, our investigation pointed us to the cleanTags function which is defined in the input.php file, which is located in the libraries/joomla/filter folder. So we thought, let’s compare the input.php file from the Joomla 3.7 website to the input.php file from the Joomla 3.6.5, and so we did, and, to our non-surprise, there were substantial differences between the two files.

Our next step (yes, you’ve guessed it!) was to replace the input.php file on the Joomla 3.7 website with one from a Joomla 3.6.5 fresh install. So we did that, and we tested the content adding (in Arabic) afterwards, and it worked, even when the content was very large! Hooray!

But why was the problem happening on Joomla 3.7?

Unfortunately, we can only say that the problem is caused by the cleanTags function (and possibly other functions called by cleanTags) in the aforementioned input.php file. We can’t give more details because we weren’t commissioned by the client to investigate the issue further (we can’t expect the client to pay for debugging Joomla errors).

We hope that you found this post useful. If you are running into a timeout when adding Arabic content (or any other UTF-8 content) to your Joomla 3.7 website, then try replacing the input.php file we mentioned above with the one from a Joomla 3.6.5 website. If that doesn’t work, then please contact us. We are always here to help, we will always find a solution, and our fees are extremely affordable!

How to Remove the Article ID from SEF URLs in Joomla 3.7

Joomla 3.7 is a weird iteration of the famous CMS – it seems that quite a few important things were half baked or not completely tested. For example, the new routing functionality that was meant to be completely revamped in Joomla 3.7 was supposedly scratched from that version, to be later introduced in a future version (Joomla 3.8?). We are saying supposedly because there were some substantial changes to the routing functionality in Joomla 3.7, which left Joomla’s routing in a twilight state between Joomla 3.6.5 and Joomla 3.8. In fact, we were surprised when we saw the code because the lead developer in Joomla clearly stated in a blog post on the official Joomla blog that the new routing code will not be shipped with Joomla 3.7. He even stated (and we’re quoting his exact words) that the code itself also had some issues, some parts are pretty much unreadable and even with debugging not easy to understand. Yet it seems that some of that unreadable code made its way to Joomla 3.7.

In any case, the new routing functionality broke the URLs on some very important websites which were relying on the sef_advanced_link configuration variable in order to remove the article ID from the URL. This is because the sef_advanced_link configuration variable now belongs to the legacy router, which will only be used when the sef_advanced configuration variable is set to 1.

So, in order to remove the article ID from the URL in Joomla 3.7, you will need to do the following:

  • Open phpMyAdmin and select the database powering your Joomla website.
  • Click on the #__extensions table.

  • Open the row where the name field is set com_content.

  • Add the following to the params field (just before the curly bracket):

    ,"sef_advanced":"1","sef_advanced_link":"1"

    Explanation: The first parameter ensures that the legacy router is loaded, and the second parameter ensures that the “advanced SEF” is used (e.g. the “article ID” is removed from the SEF URL).

  • Save the row and then login to your Joomla website and clear your Joomla cache.

  • The issue should be solved. You shouldn’t see any article IDs in the URLs anymore.

Of course, if you’re a fan of core modifications (which is something that we do not recommend), then you can do the following instead of the above:

  • Open the router.php file that is located under the components/com_content folder.
  • Change the following:

    if ($params->get('sef_advanced', 0))
    {
    	$this->attachRule(new JComponentRouterRulesStandard($this));
    	$this->attachRule(new JComponentRouterRulesNomenu($this));
    }
    else
    {
    	JLoader::register('ContentRouterRulesLegacy', __DIR__ . '/helpers/legacyrouter.php');
    	$this->attachRule(new ContentRouterRulesLegacy($this));
    }

    to:

    JLoader::register('ContentRouterRulesLegacy', __DIR__ . '/helpers/legacyrouter.php');
    $this->attachRule(new ContentRouterRulesLegacy($this));

  • Open the file legacyrouter.php which is located under the components/com_content/helpers folder, and change both instances of the following line:

    $advanced = $params->get('sef_advanced_link', 0);

    to:

    $advanced = 1;

  • Login to the backend of your Joomla website and clear the cache by going to System -> Clear Cache and then clicking on the Delete All button.

  • That’s it! The article IDs should disappear from your URLs.

We hope that you found this post useful. If you need help with the implementation, then please contact us. We are always there for you, our fees are super affordable, our work is super professional, and we always have a solution!

The Joomla Honeypot Project Experiment

At itoctopus, we are paranoid about the security of our managed clients’ websites – as such, we always research revolutionary ways to better protect these websites against potential exploits.

Last week, we conducted a honeypot project on one of the largest websites that we manage. For those of you who don’t know what a honeypot project is, it is, in essence the capture and the analysis of the malicious traffic that a website receives in order to better protect it. Typically, a honeypot project consists of a bait to lure attackers (the same way honey lures bears), but, in the case of a very large and a very high traffic website, an artificial bait is not necessary since the importance of a website is in itself a bait.

Before going into the details, we will need to explain the security setup of the website:

  • The website’s backend was protected with an .htaccess password: This method (described here) literally blocks all types of attacks on the backend. This means that we can completely forget about the attacks on the backend and focus on the attacks targeting the frontend.
  • The “index.php” file was the only file allowed to be directly executed by the Apache web server: This ensures that any attack on the website must be funneled through the index.php file first (as direct access to other PHP files on the Joomla website is blocked). This is super important since, as you will find out below, we are doing all the work in the defines.php file which is loaded by the index.php file.

  • ModSecurity was enabled on the server: This means that many known attacks are blocked before even reaching the Joomla instance. This will allow us to focus on new types of attacks.

Now that we have explained the existing security setup of the website, we can start explaining our little honepot project experiment. Are you ready? Let’s go!

Since Joomla loads the defines.php file before anything else (even before executing any serious code), we decided to add the code that will capture all the data in the requests sent to the website in the defines.php file. So, we created a defines.php file in the root directory of the website and we added the following PHP code to it:

<?php
	$file = 'project-honeypot.php';
	$projectHoneypotData = file_get_contents($file);
	$projectHoneypotData .= 'Date: '.date("Y-m-d H:i:s")."\n";
	$projectHoneypotData .= 'IP: '.$_SERVER['REMOTE_ADDR']."\n";
	$projectHoneypotData .= '--SERVER Data--'."\n";
	$projectHoneypotData .= print_r($_SERVER, true);
	if (!empty($_GET)){
		$projectHoneypotData .= '--GET Data--'."\n";
		$projectHoneypotData .= print_r($_GET, true);
	}
	if (!empty($_POST)){
		$projectHoneypotData .= '--POST Data--'."\n";
		$projectHoneypotData .= print_r($_POST, true);
	}
	if (!empty($_FILES)){
		$projectHoneypotData .= '--FILES Data--'."\n";
		$projectHoneypotData .= print_r($_FILES, true);
	}
?>

We saved the defines.php file and we uploaded it back, and then we left the website until the next day.

The next morning we checked the project-honeypot.php file, and we were amazed by the amount of garbage that hit the website: malicious referrers/user agents (that were not caught by ModSecurity), weird GET parameters, base64 POST values (some of these POST requests weren’t even relevant, since they were specially crafted for WordPress websites), and many, many attempts to upload hacked files.

The amount of data in that file was overwhelming. We had 2 options from there: either we needed to blacklist all the bad requests that we found or we needed to whitelist the good requests. We decided to go with the latter option, simply because whitelisting is a much more efficient process than blacklisting, since blacklisting means that we will need to continually monitor the website for future bad requests.

So we checked all the clean requests on the website and we whitelisted them all and blacklisted everything else (essentially, any request that was not on the whitelist was 403’d). For example, a valid POST request was one where $_POST[‘option’] = ‘com_search’ (this is sent when someone is searching for something on the website), so we whitelisted it.

Still, after whitelisting what should be whitelisted, we recorded all the whitelisted requests to the project-honeypot.php file. After 24 hours, we checked that file again, and we noticed that the absolute majority of the whitelisted requests were OK – however, there were a few bad requests among them. So, we had to blacklist those bad requests (at a granular level, any security work must include whitelisting and blacklisting – whitelisting alone is not sufficient). We repeated this process for a few days, until we were sure that almost all malicious requests were blocked from the server.

At the same time, we monitored the Apache logs for any legitimate 403s (e.g. false positives that were blocked by our whitelisting) and we did find a couple, which we immediately whitelisted.

Now, the million dollar question is, how efficient was all this?

We can safely say that the work was very efficient – we were much more comfortable with the security of the website, and we had no doubts in our minds that the website was much more secure than before. The whole experiment was so successful that we were very confident to recommend this process to our other managed clients.

Was there a performance hit caused by the security work above?

Not at all – in fact – there was a performance gain! Yes – you heard that right – there was a performance gain! And that’s because all those blocked requests were no longer loading the Joomla environment (this reminds us of a previous post of ours where we explained how not to load the Joomla environment for non existing files).

We hope that you found our post instructive and useful, and we hope that you will benefit from it to supercharge the security of your Joomla website. If you need help in doing that, then look no further. Just contact us and let us do the work for you quickly, cleanly, and affordably!

Yes – The Frontend of a Joomla Website Can Work Without the Plugins Folder

Do you have time for a simple experiment? If yes, then try the following:

  • Download Joomla.
  • Install it on a local or a remote server.

  • Once the website is working, rename the plugins folder to plugins_old.

  • Visit the website, and you will notice that the website still works.

  • Now rename the modules folder to modules_old, and you will notice that although all the modules disappeared, the website still works.

The above demonstrates the power and resilience of a Joomla instance: Joomla websites can work without the plugins (and the modules) folder(s). But, why is this worth mentioning?

It is worth mentioning because this discovery can help you in quickly debugging problems on a Joomla website without taking it offline. For example, if your website is suffering from performance issues all of a sudden, then you can rename your plugins folder to plugins_old and see if the issue persists. If it does, then the problem is not caused by one of your plugins, if it doesn’t, then the problem is definitely caused by one of your plugins. The same can be done with the modules folder.

Will the website “fully” work?

Of course, although the frontend of the Joomla website will work, it will miss all the functionalities provided by the plugins including frontend login and SEF URLs. So, you shouldn’t really do this unless you are debugging a website.

Can this be done on all Joomla websites?

No – some Joomla websites use components that explicitly require files from the plugins folders. Such websites will crash if the plugins folder is renamed.

Will the backend still work?

If you delete/rename the plugins folder, then you won’t be able to login to the backend, because you will need the authentication plugin to be able to login.

We hope that you found this post educational – if you have been reading it with the purpose of minimizing the filesystem of your Joomla website, then please note that such an endeavor is not straightforward and does not simply consist of renaming folders. Just contact us and we’ll devise a clean and sustainable solution for your needs for a very affordable price.

“Save” and “Save and Close” Buttons when Editing Articles in Joomla’s Frontend

A common issue on Joomla websites with frontend editing is that the behavior of the “Save” button on the frontend is different from the behavior of the “Save” button in the backend. The “Save” button in the backend saves the item and redirects back to the item’s page while the “Save” button on the frontend saves the item and redirects back to the listing page, which is the exact behavior of the “Save & Close” button in the backend.

This issue is particularly annoying when frontend authors/editors are writing long articles and are saving these articles frequently (they will get redirected back to the listing page, where they will need to find their article and then edit it again).

Luckily for you, our dear reader, we have a solution to this issue. With a simple trick, we were able to mimic the behavior of the backend “Save” and the “Save & Button” in a Joomla frontend. Here’s how we did it for a client of ours a few days ago:

  • We copied the edit.php file which is located under the components/com_content/views/form/tmpl folder to the templates/[ourclienttemplate]/html/com_content/form folder.
  • We then opened the edit.php file that we just created, and just before the following line…

    JFactory::getDocument()->addScriptDeclaration("

    …we added the following lines:

    $jinput = JFactory::getApplication()->input;
    $currentArticleID = $this->form->getData()->get('id');
    $editURL = JUri::root().'index.php?option=com_content&task=article.edit&a_id=' .$currentArticleID.'&return='.$jinput->get('return');

  • In the same edit.php file, and just before the following line…

    if (task == 'article.cancel' || document.formvalidator.isValid(document.getElementById('adminForm')))

    …we added the following lines:

    if (task == 'article.savestay'){
        var inputReturnURL = document.getElementById('returnurl');
        inputReturnURL .value = "";
        task = 'article.save';
    }

  • Also in the file edit.php, we replaced the following line:

    echo JText::_('JSAVE')

    with:

    echo ('Save & Close')

  • Finally, while still in the edit.php file, and immediately after…

    <div class="btn-toolbar">

    …we added the following lines:

    <div class="btn-group">
    	<button type="button" class="btn btn-primary" onclick="Joomla.submitbutton('article.savestay')">
    		<span class="icon-ok"></span><?php echo JText::_('JSAVE') ?>
    	</button>
    </div>

  • We saved the article and we uploaded it back.

  • We tested frontend editing and everything worked as it should! The “Save” button redirected back to the the same article, the “Save & Close button (as well as the “Cancel” button) redirected back to the listing page.

There is one caveat in the above trick: it only works for existing articles. If you are working on a new article from the frontend, then you must hide the “Save” button (if $currentArticleID is empty, then this means that we are creating a new article, which means that we must hide the “Save” button).

We hope that you found this post useful. If you need help with the implementation, then you can always contact us: we are always available, we are always ready, and we will not charge you much!

Performance Tip: Ensure that Missing Files Are Not Processed By Joomla

Let’s do a simple experiment together. Go to your Joomla website, type in something like: http://www.[yourjoomlawebsite].com/test.jpg (assuming you do not have a test.jpg file in the root directory of your Joomla website).

You will notice that Joomla displays a 404 error. What’s wrong with that, you may be wondering? Well, what is wrong is that the whole Joomla environment is being loaded when a simple “404 – Not Found” error would have been sufficient. Of course, on small websites, this is hardly an issue, but, on large websites with high traffic, it is.

Now the question is, why is Joomla handling all the 404s, instead of just handling the 404s related to articles not found? Well, it is because of the following code in the default Joomla .htaccess file:

# If the requested path and file is not /index.php and the request
# has not already been internally rewritten to the index.php script
RewriteCond %{REQUEST_URI} !^/index\.php
# and the requested path and file doesn't directly match a physical file
RewriteCond %{REQUEST_FILENAME} !-f
# and the requested path and file doesn't directly match a physical folder
RewriteCond %{REQUEST_FILENAME} !-d
# internally rewrite the request to the index.php script
RewriteRule .* index.php [L]

If you have some experience reading and understanding .htaccess files, you will know that the above tells Joomla to process requests to non-existing files and directories, which means that any hit on the domain with a file or a folder that doesn’t exist will eventually reach the index.php file, and ultimately load the Joomla environment. As mentioned in the article we linked to above, this can cause performance and stability issues on the server, especially on sites with lots of content and very high traffic.

So, how can one solve this problem?

Obviously the solution to this problem is to tell the Joomla environment to only execute non-file requests (we cannot block it from executing directories, because a non-existing directory can be a valid Joomla link). This can be done in the .htaccess file, by adding the following code just after the RewriteEngine On line:

# if the file doesn't exist
RewriteCond %{REQUEST_FILENAME} !-f
# and the file extension is one of the below
RewriteRule ^(.+\.(?:css|gif|jpe?g|js|pdf|php|png|xml))$ 404.php [L,NC]

Before adding the above code to your .htaccess, you will need to create the file 404.php in the root directory of your Joomla website. This file should only contain the following couple of lines (a mini-sized file ensures that the server performs minimal work and that traffic spent on non existing files is minimal):

<?php header("HTTP/1.0 404 Not Found");
die('404 - not found');?>

Once you do the above, you will notice that non-existing files will no longer be processed by the Joomla engine – which is a good thing – as this will lower the server load. If you have a large website, then the performance gain by adding the few lines of code above will be noticeable.

We hope that you found this post useful. If you need help with the implementation, then please contact us. We are always there for you, our fees are super affordable, and our work is super clean!

5 Reasons Article Saving in Joomla Takes a Long Time

Each day, more and more clients are calling us to address the slow article saving problem that they are experiencing on their Joomla websites. We’re not 100% sure why this is becoming an increasingly common Joomla problem, but we think that it might be that more large sites are using Joomla (though we have no actual statistics to substantiate our claims – we just feel it!). It might also be that we are getting more clients because of our high quality work, we don’t know for sure… But what we do know is that the emergence problem is not caused by a sudden change in the Joomla core (although, in most cases, it is caused by the Joomla core).

Over the past few months, we have worked on a countless number of websites having this problem, and so we have developed a list of all the issues that may result in long article saving time (or, in many cases, timeouts). Here they are:

1. A big assets table: The assets table is one of the most critical tables on a Joomla website – if not the most critical. A small inaccuracy in that table may crash your website. Additionally, the more data you have in that table, the slower the article saving process becomes (mainly because of the lft and the rgt fields – which we are not going to discuss in this post). A good idea is to trim that table on a daily basis in a cron job – just make sure that you don’t have any special permissions on your articles before doing that. Follow our guide here in order to cleanup the assets table on your Joomla website.

2. A lot of tags: We never liked tags – and we have to be honest why we don’t like them: it is because we think they are mostly used for webspam. Now, we have yet another reason to hate them, and it is because article saving takes a long time on Joomla websites that have many tags. If you have more than a few hundred tags on your website, then you will have a noticeable delay when you save Joomla articles. Luckily for you, the solution to this problem consists of simple core changes rather than deleting your tags.

3. Smart Search: Whoever called Joomla’s Smart Search “smart” must really have lax standards when it comes to smartness – since Smart Search is anything but smart. In fact, Smart Search is so bad that there were actual plans to remove it from the Joomla core (unfortunately, these plans didn’t materialize). In a nutshell, the problem with Smart Search is the following: it works well when it’s not needed, and it doesn’t work well when it’s needed. You see, Smart Search was developed to cater for the heavy search requirements on large Joomla websites since the normal search plugin is inadequate for large websites, however, the problem is that Smart Search doesn’t take into consideration one of the most important factors on large Joomla websites, and it is there are many articles created/updated every day. The thing is, whenever an article is saved, the Smart Search plugin runs, and splits the article into many fragments that are saved in many tables. The fragment-saving process becomes increasingly heavy with time ultimately slowing down the whole website to a crawl. The solution to this problem is simple, and it is using Sphinx instead (and, of course, disabling the Smart Search plugins).

4. The ordering algorithm of Joomla articles: Let us explain how the ordering of Joomla articles works in layman terms. Let’s say you have 9 bricks on top of each other, and you want to add, and label, an additional brick. If you want to follow the Joomla ordering algorithm for doing things, you will do the following:

  • You will ask 9 persons to help you in your endeavor.
  • You will ask each one of these persons to lift one brick, while ensuring that all the bricks are still on top of each other.

  • Ultimately, you will have some empty space at the bottom of the bricks, and so you add your brick there.

  • You label the bricks from 1 to 10, starting with the brick at the very bottom (the one that you just added).

Of course, a crazy idea would be to just add the tenth brick on top of the other 9 bricks instead of the bottom, and labeling the 10th brick “Number 10”, but where’s the fun in that? Additionally, you won’t get to meet these 9 nice people if you do it the old, unexciting way. Joomla’s ordering works in the same way, when a new article is inserted, all the articles in the same category are shifted (though not by real people) by 1. Obviously, this reordering process is very expensive on large Joomla websites. The solution to this problem is a core modification, which is explained, in details, in the article that we have just linked to.

5. A bad system/content plugin: So far, all the issues that we have discussed in this post are core issues, but the problem is not always caused by the core. Some system/content plugins, for example, try to grab data from another website using curl, a process that may take a very long time to execute and may cause the article saving process to hang. Last week, we worked on a Joomla website that had one of these bad plugins; we disabled all the system/content plugins one by one until we found it. When we opened the plugin, we noticed that it was setting the PHP maximum execution time to a very high number and was hiding all the errors. As seasoned PHP (and, of course, Joomla) developers, we know that both of these practices are very bad practices, but we also know that bad Joomla developers are everywhere.

Of course, there are other reasons that may cause the article saving process to take a very long time, two that we can think of are a hacked website and a ModSecurity rule. But they are not very common.

So, how to know which one from the above list is causing your problem? Well, you add an article to your Joomla website, and you monitor the performance of your Joomla website (or the load on your server, if you have access to that information), and if you see a degrading performance across the board (or a spike in the load), then your problem is likely caused by one (or more) of the first 4 reasons (in this case, you follow a process of elimination to know which one of the 4 is your problem). If you don’t, then your problem is likely caused by a bad plugin. Of course, if you need help speeding up the article saving process on your Joomla website, then you can always contact us. We are always there for you, our work is super clean, and our fees are super affordable.

7 Questions to Ask When Switching Your Joomla Site from HTTP to HTTPS

Earlier this week, we published a post on the importance of switching your Joomla website from HTTP to HTTPS, mainly because of a new setting in Chrome and Firefox labeling pages with sensitive forms (such as login forms) as insecure if they are not running in HTTPS mode.

Most Joomla administrators think that switching from HTTP to HTTPS simply consists of buying an SSL certificate, installing it on the server, and then forcing SSL on the Joomla website by setting the Force HTTPS field (under System -> Global Configuration -> Server) to Entire Site. Unfortunately, in most cases, the process is not that simple. You see, when you switch from HTTP to HTTPS there are many considerations that you must take into account. In fact, there are 7 important questions that you must ask yourself when you make the switch:

1- What will happen to all the social interactions?

Social interactions, such as likes, shares, etc… took place on the HTTP version of your website. This means that, by default, you will lose all these interactions when you switch to HTTPS. However, there is a way to keep these interactions, and it is by setting the data-url attribute to the HTTP version of links for the pages that were created before the switch, and to the HTTPS version of the links for all the other pages. Typically, this change is done in one plugin.

2- Is the website “301 redirecting” all the HTTP links to their HTTPS equivalents?

You must ensure that your website is redirecting (using 301 redirects) all your previous (non-secure) links to your new (secure) links. If it doesn’t, then you should address that immediately, or else you will have a duplicate content issue and your search engine rankings will drop.

3- Do all the extensions on my Joomla website fully support the switch?

Some poorly written templates/modules/plugins/components have some HTTP links that are hardcoded. For example, they request images, CSS files, and JS files from HTTP sources. This, of course, will cause a mixed content issue on your pages. The quickest way to catch the offending extensions is to thoroughly test every functionality on your website once you switch, and note, in the browser console, which images/libraries are being loaded over HTTP, and then fix the associated extensions. Ideally, you should do this on a test environment before switching.

This task is actually much more complex and important than you think it is, and we have numerous examples on how overseeing this task has caused major unhappiness.

4- Is my Joomla website displaying fatal errors on some pages because of the switch?

This is really similar to the previous point – but with a twist. Some badly written extensions may crash (or, at best, may not work properly) when you switch from HTTP to HTTPS. What makes this even more challenging is that some of these extensions may only fail under certain conditions. In order to address problems arising from such extensions proactively, you will need to closely monitor your PHP error log for the following few weeks after the move, and immediately address any errors caused by the switch.

5- Is my “$live_site” set to the HTTP version of the website?

We never recommend setting the $live_site value in the configuration.php file, and we are not very clear on why people like to set it. In fact, we have nothing good to say about this particular variable, which caused hardship for many Joomla administrators out there. However, if you feel you are compelled to set it, then make sure it points to the HTTPS version of your domain.

6- Have I disabled all SSL redirect plugins on my website?

On more than one occasion, we have fixed a problem where an SSL redirect plugin caused an infinite loop on the website as soon as the Force HTTPS setting was set to Entire Site. Make sure you disable (or, better yet, uninstall) any SSL redirect plugin that you have on your website before making the switch (they are pointless anyway and the only thing that they can do is create problems).

7- Have I checked my .htaccess file for any weird redirects?

A couple of days ago, we had a weird case of an infinite loop on a Joomla website. The infinite loop was everywhere, on both the backend and the frontend, so we disabled all the non-core system plugins on the Joomla website using phpMyAdmin, but the problem remained. It turned out eventually that the client had a redirect, in his .htaccess file, from HTTPS to HTTP, and there was another redirect in Joomla from HTTPS to HTTP. Naturally, deleting the .htaccess redirect fixed the problem.

Switching your Joomla website from HTTP to HTTPS is seldom an easy task – but worry not – we are always ready to help. Just contact us and we’ll ensure a very smooth transition of your website for very little money.

How Ordering Slows the Saving of Articles on Large Joomla Sites

Note: The solution presented in this post is a core change. Proceed carefully and always backup your website before modifying core files.

A few days ago, we received an email from someone managing a very large Joomla website (it was a Brazilian news website). She said that the saving of articles was taking an abnormal amount of time. When she first told us that, we were relieved; finally, something easy to work on! After all, it could be either an inflated assets table, or the an abundance of Joomla tags.

We started working on the task immediately, and, unfortunately, we quickly discovered that our excitement for easy work was a bit premature: it was neither the assets table nor the tags. In fact, the client regularly cleaned the assets table and the tags table was maintained at a manageable level. We were still hopeful though, so we started disabling plugins one by one – hoping that this whole issue might be caused by one of the plugins, but that wasn’t the case: we disabled all the plugins, but still we had the same problem. We were finally left with one option: we had to get our hands dirty to get to the bottom of this issue…

So, we enabled MySQL’s slow query logging and added an article to the Joomla website, and we were confident, no, make that positive, that we’ll find a very complex query in the slow query log (after all, the article saving process literally took 5 minutes), but there was nothing there, even though the long-query-time was set to 1 second (e.g. all queries taking more than 1 second were supposed to be logged there).

We were becoming more and more depressed: a task that we thought would bring joy to our hearts was becoming more and more frustrating. But then, something unexpected happened. While trying to save another article, we looked at the browser status bar (which is located at the bottom of the screen), and we noticed that the “Sending Request” part was taking a long time, and then the “Waiting” part was being executed instantly. So, this whole thing all of a sudden seemed like a DNS problem. We asked the client if she had any idea what might be causing this, and she told us that she was using Cloudflare, so we asked her to bypass Cloudflare to see if that solves the problem. She did, and then we tried to add an article, and this time, the whole thing was executed instantly. Hooray, we found the culprit, it was Cloudflare, or was it? You see, despite us feeling happy, we had this strange feeling that we didn’t nail it. After all, why would Cloudflare have this problem only with adding articles? Could it be a firewall rule being triggered on Cloudflare‘s end when someone adds an article? And, if that’s the case, then why weren’t other websites affected? Why just this one? Lieutenant Columbo would have said: “Something’s bothering me…”

So, we did another test to be sure, and this time, we had the problem again: it took about 4 minutes for an article to be saved. Our hunch was right, we didn’t find the solution to the problem… In a desperate move, we decided to debug the article saving process, so we opened the article.php file located under the administrator/components/com_content/models/ folder, and we added a die before the following line…

if (parent::save($data))

…and the die was executed instantly. We then moved the die to after the above line, and it took ages (like minutes), to take effect. So, the problem was really in the save line. Obviously, the next thing to do was to track which queries were executed at the database level by that save line. In order to do that, we debugged the problem at the database level: we printed all the queries before the save method, and then we printed all the queries after the save method (but before everything else), and it turned out that the cause was the following query:

UPDATE `#__content` SET ordering = 106 WHERE `id` = '48129'

In fact, the above query was sent to the database a few thousand times, but with a different ordering and id values. We thought that the client was lucky that saving articles was only taking 4 minutes for the whole thing (each iteration was taking about 50 milliseconds).

It didn’t take us long to find where that query was executed: it was in the table.php file located under the libraries/joomla/table folder. Specifically, the whole thing was taking place in the reorder function of that file. The obvious question at this point would be, what does this function do?

Well, the reorder function is responsible for (you guessed it) ordering articles in Joomla. For new articles, it ensures that the ordering field of the new article is set to 1, and that the ordering of all the other articles in the same category is shifted by 1. Naturally, when you have thousands of articles in each category, then the shifting process will take time.

If you’re thinking that the reorder function is poorly optimized, then you’re right: it is poorly optimized, and the Joomla core development team knows about it. In fact, Joomla versions 3.6.0, 3.6.1, and 3.6.2 addressed this problem, but, by addressing the problem they created an even bigger problem with current websites because the ordering of the new articles in the backend looked messed up (new articles appeared last), and so many Joomla developers voiced deep criticism about this abrupt change, including us. Eventually, the development team had no choice but to revert back the whole thing to the way it originally worked.

In retrospect, we believe that the ordering change in Joomla versions 3.6.0, 3.6.1, and 3.6.2 was done in good faith, but it was incomplete. While these versions fixed the performance problem by no longer shifting the ordering of existing articles by 1 when a new article was inserted, and rather just setting the ordering field of the newly inserted article to the highest ordering + 1, they didn’t address the backend views and the existing data. These versions should have modified the existing views and the existing ordering data to take the new ordering strategy into consideration. Obviously, the second part of the previous sentence (modifying the data) is very dangerous and complex, hence the decision to revert back rather than really addressing the problem.

So, how did we solve the problem?

Luckily, the client wasn’t using the Article Order ordering on any of her Category Blog menu items; she was using the Most Recent First ordering across the board. So, we were able to skip the whole ordering process by adding the following line to the very beginning of the reorder function:

return true;

The saving process worked very smoothly once we added the above line. In fact, it took our client less than a second to save articles on her website after implementing the above patch.

But, what about the ordering of featured articles on the homepage?

The client wasn’t using a menu item of type Featured Articles on the homepage, however, we understand that most Joomla websites do. So, if you want to implement the above solution on your Joomla website and your homepage consists of a Featured Articles menu item, then you will run into issues (the ordering of new articles will be messed up, and the reordering of old articles will no longer work properly). In this case, you will need to add the following lines (instead of the above line) to the beginning of the reorder function:

if ($this->_tbl != '#__content_frontpage')
	return true;

The above will ensure that the #__content_frontpage table will still be reordered by the reorder function. Keep in mind, however, that if your #__content_frontpage table is very large (e.g. has thousands of rows), then adding new articles to your homepage will take a lot of time. It’s always a good idea to ensure that this table contains a thousand entries or less (note that if you have pagination on your homepage, then this pagination will only show a thousand articles if you trim that table to a thousand rows – be very careful). We execute the following query in the cron of the Joomla sites that we fully manage to trim the data in the #__content_frontpage table:

DELETE FROM #__content_frontpage WHERE `ordering` > 1000

But why was the problem briefly solved when the client turned off Cloudflare?

The problem wasn’t solved. It was just a coincidence. When the client turned off Cloudflare, we added an article to a category that had very few articles, and so the reordering process was super quick.

We hope you didn’t find this very long post intimidating. After all, the solution really consists of adding one line (or a couple of lines) to just one file. The rest is purely detective work (or a wild-goose chase, depending on how you see things) and a discussion of the cause of the problem. Having said that, you should do the necessary due diligence before implementing the solution, mainly by ensuring that you are not using the Article Order ordering anywhere on your website. If you do, or if you are not sure, or if you just want help with the implementation, then please contact us. We will always find a solution to your ordering problem, we will ensure that your website remains stable after the implementation, and you won’t have to relinquish your hereditary rights on that beautiful south pacific island in favor of your evil half-sister to pay us our fees!

Why It’s Important to Switch Your Joomla Website from HTTP to HTTPS

Back in 2013, we have debunked the myth that switching from HTTP to HTTPS will make a Joomla website more secure, and we have explicitly recommended against using it globally. Obviously, times are different now, and while we still haven’t changed our mind that such as switch will not make a Joomla website more secure, we can no longer recommending against it. Why?

Well, because of the following reasons:

  • As of August 2014, Google considers the usage of HTTPS across the board as a minor positive ranking signal. In other words, using HTTPS on your website (the whole of your website) will give its search engine rankings a minor lift.
  • As of January 2017, Google Chrome, the world’s most used browser (in fact, at the time of writing this article, Google Chrome has more market share than all of the other browsers combined), has started displaying a “Not secure” message in the address bar when people visit a page that has a form transmitting sensitive data. So, for example, if every page on your Joomla website has a login module, then Chrome users will see that “Not secure” message on every page of your website. Not good.

  • At around the same time (January 2017), FireFox, the world’s third browser by market share, started displaying a lock with a diagonal red line to label a website matching the same criteria defined by Google Chrome as non-secure.

Unless you are sure that all of your site visitors use Bing as their search engine and Internet Explorer/Microsoft Edge as their browser (Microsoft hasn’t followed suit yet), then you must do something about it, or else your rankings will suffer considerably. Why, you may ask? Well, because when some (if not most) people see that “Not secure” message on their browser when visiting one of your pages, then they will flee your website. This will substantially increase your bounce rate, which will severely affect your search engine rankings. Yes, you’re right, the labeling used by Chrome and FireFox is inaccurate (only the data transmission is not secure, and not the whole website), but only the technical people will understand that.

So, our dear reader, if you are experiencing a higher-than-usual bounce rate on your Joomla website and you have a login form slapped on all of your pages, then the likely cause is that your website is not running in secure (HTTPS) mode, when it should. If you need help with making the switch from HTTP to HTTPS, then you are in no better place: just contact us and we’ll handle this for you quickly, efficiently, and affordably.

Unable to Save/Create Joomla Modules – What to Do?

A client called us this morning and told us that he was unable to create new modules and save existing modules. So, we logged in to his website and checked the problem, and it was like he said: we opened a module, we modified it, and then we saved it, but it wasn’t saving the new information. We weren’t also able to create new modules. What was weird though, is that we were able to set the Published status of the module from the module listing page (e.g. by clicking on the green published check mark).

At first, we thought the issue had to do with caching, but we quickly ruled out the problem when we noticed that modifications from the module listing page were taking effect immediately. In fact, the client didn’t have any caching on his Joomla website whatsoever.

We then thought the issue had something to do with a JavaScript conflict, so we checked the console in Google Chrome, but we didn’t see any errors.

We were desperate – and so we started blaming solid extensions for the problem, like JCE, for example. So we switched from JCE to the default Joomla editor, but that also didn’t fix the issue.

We were just about to do some serious debugging on the website when we noticed that the user had hundreds of groups in the Joomla ACL. Could it be like the problem we had a couple of weeks ago, where Joomla’s configuration settings were not saving, we thought?

So, we created a .user.ini file with the following content:

max_input_vars = 8000

We then placed the file in the root folder of the Joomla website, and we tried again, and this time it worked! Hooray! It was a super simple fix to a very annoying problem!

If you have the same problem on your Joomla website, then you can fix it by setting the max_input_vars to a very high number in the .user.ini file. If that didn’t work for you, or if you need help with the implementation, then please contact us. We always find a solution to any Joomla issue, our fees are super competitive, and we are always overjoyed to have good clients like yourself!

A Very Weird Joomla Issue

Back in 2011 (yes – that’s 6 years ago!) we published an article titled: “My Joomla Changes Are Not Showing!“. In short, the article stated that when Joomla changes in the backend are not visible on the frontend, then it’s a caching issue.

Of course, Joomla has changed substantially between then and now, but typically, when Joomla changes are not showing immediately, then it’s almost a certainty that the cause is caching. It might be browser caching, Joomla caching, or server (or proxy) caching – but it is, almost always, caching. We’re saying almost because on very rare occasions, the cause is not caching. Let us tell you a little story to make our point…

Yesterday evening a new client called us, and told us that she was not seeing her changes, despite clearing the cache. So, we logged in to the backend of her Joomla website, made a small change, and then checked the website, and our change did show. Naturally, we told her that her website didn’t have a problem in reflecting new changes, and we asked her to clear her browser cache. And so she did, but, even after clearing her browser cache, she didn’t see our little change. It was puzzling. We asked her to use another browser, and another machine, and she did, but the problem was still there…

Eventually, she mentioned to us that the whole problem started when her host moved her website to another server early in the morning. And so it hit us: we asked her to ping her website from her end and we pinged her website from our end, and, unsurprisingly, we both got different IPs. It seems that on her end, the website hasn’t resolved yet, but on our end, it did, and that’s why we were seeing the changes and she weren’t. Mystery solved!

Now if you, our dear reader, are not seeing your Joomla changes immediately and if you are sure that it is not a caching issue, then it might be caused by a recent move to another server, and just restarting your Internet router (and your machine) may resolve the problem. If you need help investigating the problem, then please contact us. We are always ready to help, we are always happy to help, and our prices are super affordable.

5 Reasons Why Saving Joomla Articles Is Timing Out (or Taking a Long Time)

One of the most annoying problems on Joomla websites is that saving articles can timeout (and display a 500 – Internal Server Error message) or can take an insane amount of time. This problem is not uncommon on large websites, and it typically gets worse every day. But why does this problem exist on a high profile CMS such as Joomla?

Well, there are five reasons that can cause a timeout when saving Joomla articles (or can cause the process to take a lot of time, like many seconds or even minutes):

  1. An overcrowded assets table: When you save an article on a Joomla website, a row (sometimes even more than one row) gets inserted/updated in the assets table. The problem is, the more rows you have in that table, the more costly the operation is. A “good” assets table is a table that only has a few hundred entries, more than that and this table can be the source of major trouble on your website. A quick (and dirty) fix to the “assets” problem is to clean the assets table as described here. You should also consider running a nightly cron that will clean your assets table (that’s what we do on large Joomla websites that we support).
  2. Smart Search: The so-called smart search is the unsung hero (or is it villain?) of Joomla performance nightmares. Smart search was created to replace classic search plugins on Joomla websites, and that was because the latter created performance issues on large websites. Unfortunately, not only smart search did not solve those performance issues, it added even more performance issues. You see, when you save an article with the Smart Search – Content plugin turned on, many entries (rows) will be created in the finder tables (the number of entries is proportional to the length of the article), which is a very costly process when it comes to the database. If you have millions of rows in those finder tables, then likely the problem lies in the smart search plugins that you’re using. To make sure, just disable all the smart search plugins that you have and try to insert an article. If the problem is gone, then the cause is definitely smart search. The solution to this problem is to use Sphinx instead.

  3. A system plugin: Some non-core system plugins are triggered when you save a Joomla article. These plugins are typically harmless, and they merely run only to support a feature on your website. Some of these plugins, however, perform some heavy database operations, or request a remote file (but timing out on that file because of permission reasons), thus slowing down the saving process. A good strategy to solving this problem is to disable all your non-core system plugins one by one in order to locate the rogue plugin.

  4. Shared hosting: If your website resides on the server of a mainstream shared host, then be prepared to encounter performance issues everywhere, not just when saving articles. This is because mainstream shared hosts try to cram as many websites as they can on the same server, so they’ll barely give each website enough power to run. If you are serious about your Joomla website (or any type of website), then you should go with at least a VPS. Note: avoid all these “cloud hosting” solutions – cloud hosting is technically shared hosting, but with a different, more attractive name to lure the masses.

  5. A hacked website: There are many variations of hacks on Joomla websites. Some of these hacks affect specific functionalities on the Joomla website, such as when saving new articles. Run a scan on your website (a super quick method that has very high accuracy is to run these couple of shell commands to find hacked files on your Joomla website), and ensure that you have the latest version of Joomla installed, and also ensure that all your extensions are up to date.

We hope that our post helped you find the cause (and the solution) to your article saving problems. If it didn’t, then don’t despair, just contact us and we will find the solution to the problem quickly, efficiently, and affordably.

Note: The above list is not fully exhaustive, there are other reasons (mainly having to do with the firewall or the server’s environment) that may cause article saving problems on a Joomla website.

Platform Specific Caching – A Hidden Gem Inside Joomla’s Global Configuration

Back in 2014, we presented a solution to a common Joomla problem then, where Joomla displays the mobile version of the template on the desktop (or vice versa). We stated that the problem was caused by the Joomla cache, which caches the mobile version of a page and displays it for both desktop and mobile devices if a visitor visits the website with his mobile device first (e.g. before the cache is created or after the previous cache has expired).

Admittedly, our solution was a core hack, which was the best solution then since the problem lied in the core. About 14 months after our post (on March 21st, 2016), Joomla 3.5.0 integrated our changes – almost exactly as they are, in its core. The question that you probably have on your mind right now is: “where?”. The answer is Platform Specific Caching.

Login to your Joomla website, and then click on System -> Global Configuration, and under Cache Settings, you can see Platform Specific Caching, which is set to “No” by default.

How does Platform Specific Caching works?

Platform Specific Caching works the same way our core hack works: it differentiates between the desktop cached version and the non-desktop cached version of the page by appending a string to the cache file name. The only difference is that instead of prefixing the cache file with mobile-, it prefixes it with M-. When someone from a mobile device visits the site, it will then create/serve a cache file prefixed with M-. If someone visits the site from a desktop, then it will create/serve a non-prefixed cache file.

Why is Platform Specific Caching set to “No” by default?

Because of 2 reasons: 1) not all sites have a mobile template, and 2) not all sites have a mobile template that is different from the desktop template (this is the case of websites with a responsive template, which generally don’t need to use Platform Specific Caching).

Are there any disadvantages of using Platform Specific Caching?

The only disadvantage is the fact that number of cached files will double – but, in this day and age of SSD drives with large capacity, this disadvantage is negligible.

Do websites with responsive templates need to use Platform Specific Caching?

Generally no – unless there are extensions that generate different HTML for desktop and mobile devices. In this case, it must to be used. Note that Platform Specific Caching will not cause any issue on the website (even if it’s not needed), so, if you are unsure, then you can just safely set it to “Yes”.

If your website is displaying mobile layouts on the desktop, then you should enable Platform Specific Caching. If doing so doesn’t solve your problem, then please contact us. We are always happy to serve, we are experts in Joomla, and our fees are very affordable!

On Joomla’s Core Hacks

Note: In this post, by “core file” we mean a Joomla file that cannot be overridden at the template level.

We have a somehow hawkish attitude when it comes to optimizing Joomla: we are not afraid of using core modifications (e.g. core hacks) to optimize/secure Joomla websites. For that, we are often criticized by some members of the Joomla community; these people state that it is always better to use overrides instead of core hacks. We don’t disagree, but like in everything else in life, a balance must be struck between what’s ideal and what’s realistic.

Ideally, we should have the ability to override any file on a Joomla website. Realistically, this is practically impossible, and we have an experiment that backs our statement. A few months ago, we were assigned to a project that consisted of optimizing a large Joomla website. We decided to use an extension called Joomla Override (the extension is actually referred to in Joomla’s official documentation) in order to do the optimization. At the end of the day, we discovered that many important files cannot be overridden with that extension (and any extension, for that matter), mainly because these files are loaded before anything else. By the end of the optimization mess work, we ended up with 4 overridden files, and 6 Joomla files with core modifications. Naturally, we decided that the whole thing was counterproductive, and we switched the file overrides to core modifications. A substantial part of this (failed) experiment was documented here.

But what if itoctopus developers don’t know how to properly override core files?

If you are an avid read of our blog, then you should have an idea of our level of expertise in Joomla. We know what we are talking about: there are many files that cannot be overridden on a Joomla website, and a quick example is session related files, as initial session activities must take place before loading any Joomla extension.

The only way to have the ability to override all Joomla files is for Joomla to implement the mechanism to do so. For example, Joomla can have a folder called userdefined in its main directory, and, before loading any Joomla file, it will check if a file with the same path exists under the userdefined directory. For example, if someone wants to override the includes/framework.php file, then he should create a file called framework.php under userdefined/includes. Adding such ability to Joomla is not an easy task, as all the requires / includes functions must be replaced with jrequires / jincludes functions. The latter functions will be wrappers (that must be defined in the index.php file) and will first attempt to require/include a file from the userdefined folder, and, if the file doesn’t exist, then they must require it/include it from its normal location.

Here is an example implementation of the jrequire function:

function jrequire($path, $once = false){
	if (file_exists('userdefined/'.$path)){
		if ($once)
			require_once('userdefined/'.$path);
		else
			require('userdefined/'.$path);
	}
	else{
		if ($once)
			require_once($path);
		else
			require($path);
	}
}

The above function could be written more elegantly, but you get the idea. This will allow Joomla developers to override any Joomla (PHP) file, with the exception of the index.php file in a very clean way that will sustain the test of time.

Unfortunately, at the moment jrequire and jinclude do not exist, so we have to work with what we have, and what we have (we know, that’s the third what we have in the same sentence) is core hacks, which are extremely necessary on large Joomla websites (mainly for performance reasons). Clearly, core hacks are not ideal because they get wiped out with future updates, but, if you document the changed files, then you should be OK.

Now, if you, our dear reader, would like to modify a core file that cannot be modified at the template level and you need help, then please contact us. We would love to do that for you, you will love our work, and you will love our prices!

Leveraging the Power of the “defines.php” File to Monitor POST Requests on Joomla Websites

As of version 1.6.0 (which was released 6 years ago in January of 2011), Joomla checks for the presence of a defines.php file at the main directory of the website. If it finds it, then it includes it.

The defines.php file is not part of the Joomla core, it doesn’t even come packaged with Joomla. It is, in fact, a user defined file, and not many people in the world know about it. Unfortunately, most of those who know about it associate with a horrible experience, because it was used to exploit their website (someone took advantage of their Joomla website and uploaded a malicious defines.php to their website).

For us, we think that the defines.php is an excellent tool for security. Yes – we know that the reaction to this statement would be similar to that of Professor Slughorn when Tom Riddle (aka Lord Voldemort) asked him about the Horcruxes, but let us explain…

The defines.php file is the first file to be loaded by Joomla. In fact, the code to load the defines.php is in the top of the main index.php, and not much code is executed before it (save for a few lines of code for checking if the version of PHP is OK and for getting the current time and the current memory usage). This means that it is impossible, from an application perspective, to hack the Joomla website before the inclusion of the defines.php file. This in turn means that, assuming that the website is clean, one can add additional security in the defines.php file mainly by monitoring POST requests (or any type of requests) to the website. Here’s how:

  • You create a file called defines.php at the main directory of your Joomla website.
  • In the define.php, you add the following PHP code:

    $strPostData = file_get_contents('php://input');

    The above code will grab all the data submitted using POST.

  • You check the above data for any suspicious pattern using the PHP function stripos, and, in the case of some malicious content, you just block further processing of the page using the die function.

As you can imagine, you can do a lot of powerful things when it comes to security with the defines.php file. There is one caveat, however, is that the code in the defines.php will only intercept POST requests made to the main index.php file. So, if you have an additional PHP file that is directly accessible by the outside world, then the defines.php code will not be executed for that file. If you want to ensure that all traffic goes through your defines.php filter, then you should ensure that the index.php file is the only PHP file directly accessible by the outside world.

So, was additional security the original purpose behind the defines.php file? If yes, why wasn’t this additional security added to the Joomla core?

No. Additional security was not the original purpose. The defines.php concept was created by Joomla developers to store user defined global constants. For example, if some of your extensions use the now obsolete DS constant, then you can add the following to your defines.php file:

define('DS', '/');

The above line will ensure that any DS constants are properly replaced by the forward slash (/).

Do you need to place code in the defines.php file inside PHP tags?

Definitely! The defines.php must be treated like any other PHP file. You must have the opening and closing PHP tags.

We hope you found this post useful, and we hope that you take advantage of the defines.php file in order to strengthen the security of your Joomla website. If you need help doing so, then please contact us. We are always there for you, our prices are super affordable, and we are experts in Joomla security.

Joomla Configuration Settings Not Saving? Read This!

A curious thing happened to us this morning while trying to update a supposedly small Joomla 2.5.28 website to 3.5.1. No – we didn’t suddenly develop xray vision, it was something even curiouser than that!

Let us begin by explaining what we were doing… We were, as stated before, trying to update a small Joomla website from 2.5.28 to 3.5.1 by following the official guide (we only do that for very small websites – since it expedites the work – otherwise, we perform the migration manually). As you may probably know, the second step of a migration from 2.5.28 to 3.5.1 (the first step is always the backup step), would be to change the Update Server from Long Term Support (recommended) to Short Term Support in the Options of the Joomla Update extension, but, when we tried to do that on the client’s site, we saw something unexpected: we saw the Joomla configuration settings page loaded in the popup after clicking on the Save button! Yes – you heard (or read) that right: the popup loaded the configuration settings page! Huh?

That was very weird, we thought. We thought it was a glitch, and so we did it again, and again, and again. Still, we had the same problem, but we did notice something: we noticed that the popup had a large number of fields inside it, mainly because there was a huge ACL database on that particular website. Could it be, we thought?

We then went to the configuration settings page, and made a small modification (we decreased the session lifetime from 60 to 59), and then tried to save it. Unsurprisingly, our little change wasn’t saved! We knew immediately what was happening: the number of the fields on that page (and the Joomla Update popup) was exceeding the maximum number of inputs allowed by the PHP instance on the development server (we were doing the update on a development server).

To verify our theory, we checked the value of max_input_vars (by creating a small PHP page with phpinfo(); in it), and it was set to the default value of 1000. Checking the HTML code of the Joomla configuration settings page revealed that the page had the following fields:

  • 2933 select fields
  • 583 input fields
  • 4 textareas

The total number of fields was 3520, which was more than triple the PHP limit. Clearly, that was the problem!

So, how did we solve the problem?

We solved the problem by creating a .user.ini file in the main directory of the Joomla website (e.g. at the same level of the index.php file) with the following content:

max_input_vars = 8000

As you can see, we increased the maximum number of input variables to 8,000, which is more than double the current need. We then tried to modify a setting in the configuration settings, and this time it worked!

Will the above solution always work?

The above solution should work flawlessly on most recent servers (especially those using WHM). However, on some older servers (or servers with weird environments), it may not work. If it doesn’t, then try renaming the .user.ini file to php.ini file. If it still doesn’t work, then create a file called defines.php in the main directory of your Joomla website, and then add the following code to it:

<?php ini_set('max_input_vars', '8000'); ?>

If the above still doesn’t work, then you will need to modify the max_input_vars value in the master php.ini (e.g. the server’s php.ini file) and then restart Apache. If you are on a shared hosting, then most likely you don’t have access to that file, which means that you will need to beg ask your hosting company if they can do it for you.

How come the client didn’t notice the problem?

The client didn’t notice the problem because the production website resided on a different server with an already altered max_input_vars. Obviously, whoever was working previously on that Joomla website noticed the problem before and addressed it on the production server.

Are there any side effects for having a large max_input_vars value?

Yes – DoS (Denial of Service) attacks – especially ones using hash collisions (aka HashDOS attacks) – will be more dangerous on servers where the max_input_vars directive is set to a high value. This problem, of course, is more relevant to websites with high exposure and many enemies (as these are the websites that typically get hit with more than the FDA recommended intake of DoS attacks).

But why so many fields in the configuration settings page?

Well, there are 2 entities that must be blamed here: the Joomla administrator (that’s you), and Joomla itself. First, if a Joomla website has more than 100 groups and/or access levels, then there must be something conceptually wrong with the website. In other words, the groups/access levels are being used for the wrong purpose. Second, Joomla must not display all the ACL data on one page – not only this is a problem at the server level (where the max_input_vars can be exceeded), but it is also a problem at the client level (the browser may crash because of all these fields). Additionally, Joomla must have a mechanism to check for the max_input_vars and warn the administrator if a certain page is exceeding the effective limit.

We hope you enjoyed reading this post as much as we have enjoyed writing it. We have tried our best to make it as uncomplicated as we can. We reckon, however, that non-developers may find it a bit over their head. If this is the case for you, then please contact us. We are always happy to serve, our fees are affordable, our work is top notch, and we always strive for the cleanest solution possible.

How to Add a Joomla Super User from phpMyAdmin

Ever since the elevated permissions (or elevated privileges) exploit in Joomla versions less than 3.6.5 was made public, we are seeing weird things happening to the #__users table on hacked Joomla websites. These weird things can be any (or all) of the following:

  • The usernames of all the users are changed to admin.
  • All super users are deleted.

  • New super users are created.

In this post, we will address the second problem – when all super users are deleted. We will explain how to create a super user from scratch using phpMyAdmin:

  • Login to phpMyAdmin.
  • Select the database that is used for your Joomla website. You can know which database is used on your Joomla website by checking the $db value in the configuration.php file.

  • Click on the SQL tab, and then run the following queries (replace #__ with your database prefix, which is the value of the $dbprefix variable that can be found in your configuration.php file):

    INSERT INTO `#__users` (`id`, `name`, `username`, `email`, `password`, `block`, `sendEmail`, `registerDate`, `lastvisitDate`, `activation`, `params`, `lastResetTime`, `resetCount`, `otpKey`, `otep`, `requireReset`) VALUES (1000000, 'tempsuperuser', 'tempsuperuser', 'youremail@yourjoomlawebsite.com', MD5('temppassword'), '0', '0', '0000-00-00 00:00:00', '0000-00-00 00:00:00', '', '', '0000-00-00 00:00:00', '0', '', '', '0');

    INSERT INTO `#__user_usergroup_map` (`user_id`, `group_id`) VALUES ('1000000', '8');

  • That’s it! You should now be able to login to your Joomla website as super user using the username tempsuperuser and the password temppassword. Try it!

Does the above guide guarantees that the super user won’t be deleted in the future?

No – it just recreates a super user from scratch. If you see weird activity in your #__users table, then most likely your website is hacked/vulnerable and it is very possible that you will see the same issue in the future. You will need to update your Joomla website to the latest version, ensure that your extensions are all up-to-date, and follow our important Joomla tips here.

If the above guide is a bit over your head, or if you need help in securing your website, then please contact us. We are always here to help, we work 24/7/365, our fees are super affordable, and we are the friendliest Joomla developers on this planet (and many other uninhabited planets)!

Module Title Is Not Showing on Joomla: How to Fix

A new client emailed us this morning and told us that the titles of modules assigned to a certain position are not showing. He told us he was using the latest version of Joomla, and that he ensured that “Show Title” was set to “Show” in the module’s settings. He told us that he’s confident that it was not a caching problem, since he was not using any type of caching and all of his other changes were showing immediately… “What could it be?”, he desperately asked us…

Luckily, we have seen this problem many times before… The problem is neither caused by caching nor by the module itself – the problem is caused by the template. Let us explain…

When you want to create a new module position on your website, you add something like the following code to your template’s index.php:

<jdoc:include type="modules" name="module-position" type="xhtml" />

The above ensures that any enabled module assigned to module-position will be displayed on the pages it is assigned to. Additionally, the above ensures that the styling of the module is set to xhtml, which means (among other things), that the module’s title will be displayed.

However, if you add something like the below to your template’s index.php

<jdoc:include type="modules" name="module-position"/>

…then the default styling will be used for the modules assigned to module-position, and that default styling is none (note that Joomla’s official documentation erroneously states that the default styling is table – this is wrong – the default styling is none). The none default styling means that the title will not be displayed, and no preset styling will be applied to the module.

As you may have probably guessed when reading this post, our client used something like the latter line in his template, and that’s why the module title was not showing. Adding type=”xhtml” to the module declaration in the template fixed the problem…

If you have the same problem on your Joomla website, then make sure you add type=”xhtml” in the module’s declaration in your template (or make sure you replace type=”none” with type=”xhtml”). Once you do that, clear your Joomla cache and see if it works. If it doesn’t, then please contact us. We will help you fix the problem in no time and for a very affordable fee.

“To use reCAPTCHA you must get an API key” Error when Using reCAPTCHA on RSForm

A client of us told us that he didn’t like the default CAPTCHA (by the way, in case you really want to know, CAPTCHA stands for Completely Automated Public Turing Test to Tell Computers and Humans Apart) field that comes with RSForm. He said that the letters looked very small, and that it affected his conversions.

He asked us to switch all his forms to use reCAPTCHA instead, and so, we downloaded and installed the reCAPTCHA RSForm plugin on the Joomla website, we enabled the plugin, and then we went through each and every one of his forms, and switched all the CAPTCHAs to reCAPTCHAs.

While having the strange feeling that we missed something, we went to one of the forms to see if it’s working on not, but instead, we saw the following message on the very top of the page…

To use reCAPTCHA you must get an API key from http://www.google.com/recaptcha/whyrecaptcha

…and, in the place where the CAPTCHA should have been displayed, we were seeing the following error:

Input error: k: Format of site key was invalid

Aha, so our instincts were right, we did forget something: we forgot to generate a reCAPTCHA API key, and so we went to the above URL (while logged in to our Google account), and followed the instructions to generate a Site Key and a Secret Key. Once we did that, we went back to the Joomla website, we enabled the Captcha – ReCaptcha core system plugin, and we added our Site Key and our Secret Key in its settings.

We then went to an RSForm form on the website, but still, we saw the same error… That didn’t make sense, we thought… How come we are still seeing the same error when we just added the API keys to the core reCAPTCHA plugin. Logically, the next step was to find out how the above error was triggered, and so we searched the Joomla files for the following line: To use reCAPTCHA you must get an API key.

The search revealed that the error was thrown by the function rsform_recaptcha_check_answer in the file relib.php (which was located under the plugins/system/rsfprecaptcha folder), and it was thrown because $privkey (which is reCAPTCHA‘s Secret Key) was empty. So we checked how this variable was passed to the function, and it turned out that the plugin was grabbing this value from the settings of the RSForm extension, which was obvious in following line (the line was located in the rsfprecaptcha.php file, which was in turn located in the plugins/system/rsfprecaptcha folder):

$privatekey = RSFormProHelper::getConfig('recaptcha.private.key');

Aha! So RSForm had its own reCAPTCHA settings, and so it wasn’t using the settings of the core reCAPTCHA plugin. So we logged in to the backend, and then we clicked on Components -> RS Form Pro -> Configuration, and we found a tab called reCAPTCHA. We entered our Site Key in the Public Key field and our Secret Key in the Private Key field, we saved the settings, and then we visited the form and this time it worked! Hooray!

Now if you, our dear reader, are seeing the above error on your website, then make sure that you enter reCAPTCHA‘s Site Key and Secret Key in RSForm‘s reCAPTCHA settings. If, after doing that, you are still seeing the same error, then please contact us. We will solve the problem for you quickly, affordably, and professionally!

Registration Form Redirects to Login Form on Joomla 3.6.5

While processing what seemed to be a very simple request from one of our clients this morning, we encountered a very weird issue. The simple request was that the client wanted a simple registration form on his website, and the weird issue was that, for an unknown reason, we were redirected to the login form everytime we tried to visit the registration URL.

Here’s what we did:

  • We created a new menu item of type Registration Form.
  • We set the alias of that menu item to be register.

  • We visited http://www.[ourclientjoomlawebsite].com/register, but we were redirected automatically to http://www.[ourclientjoomlawebsite].com/register?view=login, which is the login page.

We checked everything – we checked whether the client had sh404SEF installed, and he didn’t. We checked the .htaccess file for weird redirects, but the client was using a very standard .htaccess file.

After a lot of investigation, we discovered that the root cause of the problem was that user registration was disabled on the website. We did the following to solve the problem:

  • While in the backend of the Joomla website, we clicked on Users -> Manage (which displayed a listing of all the Joomla users).
  • We clicked on Options on the top right.

  • We changed the value of Allow User Registration from No to Yes.

  • We saved the settings and then we tried again.

  • This time it worked!

Joomla was doing the right thing by redirecting the registration page to the login page, but, in our opinion, Joomla must have warned us when we created the menu item of type Registration Form that Allow User Registration was disabled. Had it done so, it would have saved us a lot of time!

In any case, if you, our dear reader, are having the same problem on your Joomla website, where the registration page is redirecting to the login page, then make sure that Allow User Registration is enabled on your website. If it is, then try using another browser (some browsers temporarily save redirects) and see if the problem is fixed. If it still isn’t fixed, then please contact us. Our fees are super affordable, our work is super professional, and we are the friendliest developers on planet Earth.

All URLs Display Homepage Content on a Joomla Website Powered by NGINX

We had a weird case late last week: a new client contacted us and told us that all the links on his website were pointing to the homepage. He told us that he was using NGINX, and he told us that he thought that NGINX is the cause of the issue.

So, we checked the website – it wasn’t that all the links were pointing to the homepage, it was that all the links were displaying the homepage. For example, http://www.ourclientjoomlawebsite.com/ and http://www.ourclientjoomlawebsite/link-1 and http://www.ourclientjoomlawebsite.cm/link-2 were all displaying the contents of the homepage. The links were not being redirected, the links were just displaying the contents of the homepage.

The first thing that we did was to empty the $live_site variable in the site’s configuration.php file. So, we changed:

public $live_site = 'http://www.ourclientjoomlawebsite.com';

to:

public $live_site = '';

Unfortunately, that didn’t work, but, deep down inside, we didn’t expect it to: it was a long shot…

The second logical thing to do was checking the nginx.conf, which is the NGINX configuration file. Here’s what we did:

  • We ssh’d to the server as root.
  • We opened the file nginx.conf which is located under the /etc/nginx folder.

  • We checked if the location / statement existed in the code (which is needed for Joomla to work correctly under NGINX), and it didn’t, so we added the following code immediately before the last line (the last closing curly bracket):

    server {
      location / {
         expires 1d;
         try_files $uri $uri/ /index.php?$args;
      }
    }

  • We saved the file, and then restarted the NGINX webserver by issuing the following command in the shell:

    /etc/init.d/nginx restart

    Note: If after restarting nginx you see the following error: Restarting nginx (via systemctl): nginx.serviceJob for nginx.service failed because the control process exited with error code. See “systemctl status nginx.service” and “journalctl -xe” for details., then the problem might be that you didn’t put the location code block inside a server code block.

  • We tested the website, but still, the same problem…

Odd, we thought… Mind you, disabling SEF did solve the problem, but, naturally, that wasn’t a real solution and it wasn’t an option for the client…

After spending a few hours beating around the bushes, we started pondering about exploring cheap suicide tactics, like holding our breath for too long. Luckily, it didn’t come to this, as we saw something curious: when we visited the administrator section of the website, we noticed that no CSS style was applied to the login page, which was weird. Checking the HTML code, we noticed that the following file was requested:

<link href="/templates/isis/css/template.css?de6fd0cafae3e6687ab8a1abd290a957" rel="stylesheet" />

But, from our experience, the above line should be something like:

<link href="/administrator/templates/isis/css/template.css?de6fd0cafae3e6687ab8a1abd290a957" rel="stylesheet" />

So, why is administrator missing from the link? Is it related to the original problem? It must be related to the original problem – because it’s a substantial issue with the URL, which is exactly what the original problem is.

The missing “administrator” problem restored our hopes, because not only we were increasingly confident that it was directly connected to the original problem, we were also confident that we were able to fix it, as the problem was easily traceable through code.

Once we investigated this issue further, we discovered something interesting, the JURI::Base() function was always returning empty. Huh?

So, we checked the implementation of the JURI::Base() function in the file uri.php file which is located under the /libraries/joomla/uri/ folder, and we discovered something even more interesting, $_SERVER[‘PHP_SELF’] was empty! In fact, it was always empty!

Joomla relies on $_SERVER[‘PHP_SELF’] heavily in its handling of SEF URLs and in its backend, which means that the original problem is really caused by $_SERVER[‘PHP_SELF’]. We found the root cause of the problem! Hooray! The next step was to actually solve the problem…

It was obvious that the problem was caused by the NGINX web server, and so we made an extensive research about the issue, and we discovered that, in order to address the problem of the empty $_SERVER[‘PHP_SELF’], a certain PHP global variable, called PATH_TRANSLATED, must be explicitly defined in the fastcgi_params file that is used by the nginx.conf file. The fastcgi_params is typically found in the /etc/nginx folder (at the same level of the nginx.conf folder).

Another thing that needed to be done was to change the value of cgi.fix_pathinfo in the php.ini from 0 to 1.

Here’s how we did all the above:

  • We added the following code to the nginx.conf file (instead of the code added near the beginning of this post):

    server {
      location / {
    	expires 1d;
    	try_files $uri $uri/ /index.php?$args;
    	fastcgi_split_path_info ^(.+?\.php)(/.*)$;
        if (!-f $document_root$fastcgi_script_name) {
    		return 404;
        }
    	fastcgi_param HTTP_PROXY "";
    	fastcgi_pass 127.0.0.1:9000;
    	fastcgi_index index.php;
    	include fastcgi_params;
      }
    }

  • We opened the file fastcgi_params (again, which is located under the /etc/nginx ) folder and we added the following line (after the PATH_INFO line):

    fastcgi_param  PATH_TRANSLATED    $document_root$fastcgi_script_name;

  • We restarted NGINX by issuing the following command:

    /etc/init.d/nginx restart

  • We opened the php.ini file (please check phpinfo(); to know the exact location of your working php.ini file) and we changed the following line:

    cgi.fix_pathinfo=0

    to:

    cgi.fix_pathinfo=1

  • We restarted php-fpm by issuing the following command in the shell:

    /etc/init.d/php7.0-fpm restart

  • We checked the website and the problems, all the problems, were solved! The URLs were working and the administrator section was working! Double hooray!

We are not sure whether this problem applies to all NGINX servers, or to just NGINX servers using PHP 7. In either case, the official Joomla documentation about NGINX should be updated, as it no longer reflects a guaranteed working solution.

If you’re running NGINX and your website is displaying the homepage for all the URLs, then please follow our guide above. It should work for you as it worked for us. In the unlikely case where it doesn’t work for you, then please contact us. Our service is fast, our work is professional, we are reliable, and our fees are super affordable.

Out of Memory Error vs Allowed Memory Error on a Joomla Website

We know, it’s been a couple of weeks since we last wrote a post on this blog – we’re not getting lazy, it’s just that we had quite a few large projects in December. Still, we feel guilty, and so we decided to write a post right now, and if you want to know what right now is, then it is 8:52 PM EST on December 31st, 2016. Yes, it’s new year’s eve, and we’re writing a blog post!

OK – enough with this introduction. Let’s start with the juicy stuff…

As a Joomla administrator, you may have seen one of the following errors on your Joomla website (or maybe you have seen both of them):

Fatal error: Allowed memory size of x bytes exhausted (tried to allocate x bytes)

Fatal error: Out of memory (allocated x) (tried to allocate x bytes)

To the untrained eye, the above errors look more or less the same: they look like a memory problem, and this is correct, they are memory issues. But, they are completely different memory issues. Unfortunately, most Joomla administrators think that the above errors are the same, and they think that the solution presented in this post will fix both of them. This isn’t the case…

In fact, increasing the memory_limit in the php.ini or the .user.ini file may address the first problem, but it certainly will not address the second problem. You see, the first problem means that the Joomla website has exceeded the allowed memory limit for the PHP instance, while the second problem means that the Joomla website has used up all the available memory. Let us give you an example…

Say your default php.ini has a memory limit of 32 MB, and say you have 2 GB of RAM on your server… If a page on your Joomla website exceeds 32 MB, then you will see the Allowed memory… fatal error. However, if the memory limit in your php.ini file is set to a very high number, like 4096 MB (or 4 GB, which should never be the case), and you have 2 GB of RAM on your server, then if a page on your Joomla website needs more than 2 GB, then you will see the Out of memory fatal error.

Now, if you read the above paragraph very carefully, then your question would be, why would anyone, in a real life scenario, see the Out of memory fatal error? Good question!

In fact, in a real life scenario, the Out of memory fatal error is never thrown because all of the server’s memory was exhausted, it is thrown because a hard limit set at the Apache level is exceeded, and that hard limit is set using the RLimitMEM directive in Apache. For example, having the following in your httpd.conf file…

RLimitMEM 33554432

…means that there is a hard limit of 32 MB on the memory usage on your Joomla website, which means that any page on your Joomla website needing more than 32 MB will throw the Out of memory fatal error (assuming the PHP memory_limit is higher than 32 MB, otherwise, the page will throw an Allowed memory fatal error).

So, how to address an “Out of memory” fatal error?

Obviously, addressing such an error can be done by either commenting out the whole RLimitMEM line in the httpd.conf file, or by increasing the value of RLimitMEM (also in httpd.conf) to a higher value, and then restarting Apache. Note that if you’re using WHM, you can change the value of RLimitMEM from within WHM, here’s how:

  • Login to the WHM console as root.
  • In the search box on the top left, search for Apache Configuration.

  • Click on Apache Configuration.

  • Scroll down a bit and click on Memory Usage Restrictions.

  • In the Configure Apache RLimits page, you can either disable RLimitMEM or you can just set it to a higher value. Note that you will need to restart Apache for the settings to take effect.

If you’re seeing the problem and you are on shared hosting, then you should contact your hosting company, and they may increase it for you (but don’t hold your breath, as they most likely won’t).

If the above doesn’t fix the problem, then please contact us. We are always ready to help you (even on new year’s eve), our fees are super affordable, and our work is top notch!

On a final note, we would like to wish you a Happy New Year 2017, may it be full of happiness, health, and prosperity for you and your family!

Disqus Account Hijacked – What to Do

Let us tell you a little story…

A huge Joomla website that we fully manage uses Disqus for commenting. The website receives an insane amount of traffic and it is in the Alexa top 10K websites in the US. About 4 months ago, we noticed that the Disqus commenting section had irrelevant ads, but we thought that the client authorized these ads, and so we dismissed this as a non-issue.

Two months ago, the client contacted us and complained about these ads, and so we told them that we were under the impression that these ads were authorized from their end, the client responded that this wasn’t the case, and so we delved deeper into the mysterious world of Disqus.

After investigating the issue, we discovered that Disqus had a platform for displaying ads called “Reveal” which was made very aggressive beginning of the year (beginning of 2016). “Reveal” pays money to publishers, and it was paying the Disqus account owner (who was a developer who worked on the website back in 2011, before we started managing it) about $3,000/month by ACH (ACH stands for Automated Clearing House, another term for direct deposit). Naturally, we disabled “Reveal” in Disqus in order to stop this madness.

We then urged our client to do one of the following:

  • Switch to another Disqus account that they own in order to prevent this in the future, and then ask Disqus to copy the data from one account to another. This option was ideal, but we had serious doubts on whether Disqus will accommodate this request; why should they? It is a free service after all and no one should expect support for anything that was given away for free.
  • Talk to the previous developer and ask him to surrender his admin rights on the account. We also had doubts about this – a developer who sneaks in his information to make a quick buck out of a very reputable client is not a developer that will give away such things easily (or cheaply).

  • Use a Joomla extension such as JComments for commenting platform and then copy the data from Disqus to JComments. This suggestion was the weakest, in our opinion, but it was the only one that was guaranteed to work. Disqus has a solid API, and that API can be used to build the migration tool.

After discussing the issue internally for a few days, the client told us that they will go with the second option, and so we closed the issue on our end. However, we did have this little gut feeling that we will hear from the client about this particular issue in the near future.

And so we did: a couple of weeks ago, the client called us and told us that they were seeing those ugly ads again, and that they needed them removed. So, we tried to login to Disqus, and this time, it didn’t work for us. We asked the client whether they changed their username and/or password on Disqus, but they said they didn’t. We understood what happened immediately: the previous developer discovered that he didn’t receive that $3K in the previous month, so he re-enabled “Reveal” (the Disqus advertising platform, in case you just skipped to this paragraph), and he changed the client’s Disqus password. That was nasty but it was expected.

So, we presented the client with the above 3 options again, and they told us that they tried contacting the previous developer, but he didn’t reply. So, we asked the client to see if they can contact Disqus about this (we also told them to expect very little), and we told them that meanwhile, we will migrate their Disqus comments to JComments.

Immediately after we finished the Disqus migration script to JComments, the client contacted us and told us that, to their (and our) surprise, Disqus replied to them about this and they were willing to help. Here is the email that Disqus sent to our client:

Hi,

Thanks for reaching out to us.

If you have lost your moderator login information, or no longer have access to your forum’s moderation panel due to changes in ownership, you will need to provide the following information so that we can update the Primary Moderator:

-Evidence that you own the site or are in control of the site (this could be a page created on the site with a Disqus-related message, or a screenshot of the site admin or domain registration).

– Shortname of the forum being modified.

– Registered Disqus username of the desired new primary moderator (if you do not have an account you may register one at http://disqus.com/profile/signup ).

– Registered email address of new primary moderator.

If your situation demands more than being set as the Primary Moderator for the forum and having access to the Moderation Panel, please provide us with a detailed description of the issue you are facing.

Woohoo! Disqus did beat our expectations (and those of our client), and they only requested from us to create a specific page for them in order to prove ownership of the website. So, we quickly created a page called “Hi Disqus!” which contained the following text:

“This is a message for Disqus. Thank you for giving us back control over our Disqus account.”

We also created a Disqus account for our client (using an email address belonging to our client’s domain), and we forwarded the account information to our client, telling them to ask Disqus to make the account the master account.

In less than 24 hours, Disqus did just that! They were unbelievably helpful and they were not as we thought they were. We were surprised that they even responded, let alone granting our client full control over his Disqus comments.

We were thrilled that this whole thing ended in the best way possible, and so was our client.

Now if you, our dear reader, lost control over your Disqus account (or didn’t have full control in the first place), then we suggest that you contact Disqus (they will reply to you). If you need help with the process, then please contact us. We have done this before, our fees are super affordable, and we will do our best to turn that nightmare of yours into a positive experience.

List of Website Types That We Are Unable to Work On

Frequently, we get calls from potential clients asking us to fix problems on their Joomla websites, only to find out afterwards that we are unable to do any work for them because of the nature of their websites. Clearly, this leads to disappointment on both ends: the clients need our work and they can’t have it, and we need the clients’ money but we can’t have it.

In order to minimize the disappointment on both ends, we are publishing the list of websites that we cannot work on. Without further ado, here’s the list:

  • Websites with adult material, including nudity websites.
  • Websites selling alcohol and/or promoting the use of alcohol.

  • Websites related in any way to gambling/betting. This includes websites primarily discussing gambling/betting tips and tricks, gambling/betting websites that do not use real money for transactions, and websites primarily selling products that are used for gambling/betting.

  • Websites selling narcotics and/or promoting the use of narcotics. This includes websites selling and/or promoting cannabis for medical or non-medical purposes.

  • Websites with extreme political content (e.g. political websites that incite hate).

  • Websites used for scamming visitors.

  • Websites knowingly and intentionally selling counterfeit products.

  • Websites glorifying the suffering of any living thing (living thing may be an oxymoron, but you get the idea).

  • Websites selling (and/or promoting) magic services and/or products.

  • Websites with obscene content.

  • Websites intentionally promoting fake and/or misleading content.

  • Websites belonging to organizations and/or countries that are on the US or the Canadian embargo list.

The list above reflects our solid values that we believe will promote a better and cleaner Internet (and world) for everyone (we hope that all other developers follow suit, but we know that it’s almost impossible, after all, money is insanely seductive to the absolute majority of developers out there).

If you are in doubt whether your website falls into our blacklist or not, then please contact us, and we’ll give you a definitive and straight answer.

A Script to Migrate Articles’ Meta Keywords to Joomla Tags

Note: Before you use our script, please backup your website (filesystem + database). While we have tested our script many times it’s always better to be on the safe side and backup your Joomla website first.

Important note: The script is provided “as-is”, we can’t claim responsibility for it. We can’t even claim that it will work for you – in fact, running it may even turn on and off the lights in your house incessantly until a 33 year old man who speaks 7 languages (with Gibberish as his first language) says the word “itoctopus” correctly! You have been warned!

We don’t think highly of tags, especially Joomla tags. In previous posts on this blog, we mentioned why they were bad from an SEO perspective and from a technical perspective. Nevertheless, many of our clients use them, and so we cannot ignore the fact that they are there.

In fact, quite a few Joomla administrators were using tags even before Joomla integrated a tag system into its core. Some of them used extensions for tagging, others used meta keywords in articles and simulated a tagging system on their Joomla website.

Naturally, almost all of these Joomla administrators wanted to switch to Joomla’s own tagging system ever since it was introduced, but it wasn’t an easy task: the switch was technically a (costly) migration from one system to another. So, the absolute majority of them kept on using what they were using, with a dream that one day, someone, somewhere, will create a tool that will migrate their tags to Joomla’s tagging system.

Well guess what? For those using meta keywords for tagging the someone is itoctopus and the somewhere is the charming city of Montreal: we have created a script that will smoothly migrate the meta keywords of Joomla articles to tags. Yes, we’re not kidding, you can download the script from here!

All you need to do is to extract the zip file, and copy the unzipped PHP file called migrate-meta-keywords-to-joomla-tags.php to the root directory of your Joomla website (e.g. at the same level of your index.php file), and then point your browser to http://www.[yourjoomlawebsite.com/migrate-meta-keywords-to-joomla-tags.php. Once you do that, the script will automatically generate tags out of your meta keywords and assign them to the appropriate articles. That’s it, you don’t have to do anything else! Convenient and simple, isn’t it?

So, how does the script work?

The script grabs all the articles (that have meta keywords) from your #__content table, and then it loops through each entry, generating tags using a modified createTagsFromField function (this function exists in the tags.php file which is located under the libraries/cms/helper folder), and tagging articles using the tagItem method of the JHelperTags tags class.

Will the script create redundant data if it’s run more than once?

No – it won’t. If you run the scripts multiple times, then the existing tags won’t be recreated, and the articles will be untagged and retagged. In other words, you won’t end up with duplicate data if you run the script multiple times.

How long will the script take to execute?

It depends on the number of articles with meta keywords that you have on your website. We found that it takes around 1 minute to process 400 articles on a medium power server. Note that the more tags you have in the #__tags table, the slower the script will be (e.g. the second batch of 400 articles will take a bit more than a minute to process, and the third batch will take more than the second batch, etc…)

Will the script timeout if there are too many articles with meta keywords?

In most cases, it shouldn’t. We have ensured that there are no limits on the amount of time that the script can take. Nevertheless, some server settings (especially on cheap hosts) disallow any alteration of script timeout limit. If this is the case for you, then you will need to use the LIMIT MySQL clause in the main query to ensure that you process the articles in batches (we have explained how in the script).

Will too many tags cause the Joomla website to be slow?

Definitely. Especially on the backend when you are trying to create/edit articles. One of the posts we have linked to at the very beginning of this post explains this issue in details and how to overcome it.

Can I use this script anywhere for free?

Yes – it’s free for both personal and commercial projects. No need to alert us if you want to use the script. But, if you need help using it, then you can always contact us, but please note that our super affordable fees apply.

Empty (Not Blank) Joomla Backend – Cause and Fix

Note: Any reference to ‘#__’ in this post must be replaced with the database alias of your website, which is defined in the configuration.php file.

This noon, a client called us and told us that whenever he was trying to login to his Joomla website, he was seeing an empty page. He was quick to point out that by empty page he meant a page with virtually no menus and no modules, and not a blank page (which is the sign of a fatal error). He said he was only seeing the Joomla logo, the notification messages module, and the copyright at the bottom of the page. In short, the client was seeing something exactly like the below (we changed the name of the client’s website to Joomla):

Empty Joomla Backend

Figure 1: Empty Joomla Backend

Luckily, we saw this error before, and we remember it had to do with either a corrupt #__assets table or a corrupt #__viewlevels table. We weren’t completely sure (unfortunately, we haven’t documented the solution on our blog).

Since the #__assets table is much larger and much more complex than the #__viewlevels table, we decided to start by checking whether the #__viewlevels table is corrupt or not. So we logged in to phpMyAdmin and we compared the table to the one of a fresh Joomla install, and we noticed the following couple of differences:

  • There was one additional row in the first table (e.g. that of the client).
  • The rules were not identical for the corresponding rows.

We immediately concluded that the problem was a corruption in the #__viewlevels table, and so we replaced the data of in the first table with that of the fresh Joomla install the following way:

  • We truncated the table by issuing the following SQL command in phpMyAdmin:

    TRUNCATE TABLE `#__viewlevels`;

  • We added the clean data by issuing the following query:

    INSERT INTO `#__viewlevels` (`id`, `title`, `ordering`, `rules`) VALUES
    (1, 'Public', 0, '[1]'),
    (2, 'Registered', 2, '[6,2,8]'),
    (3, 'Special', 3, '[6,3,8]'),
    (5, 'Guest', 1, '[9]'),
    (6, 'Super Users', 4, '[8]');

  • That’s it!

We then tried to login to the website, and this time it worked. Hooray!

So, what caused this problem?

We think that either the client or someone working for the client mistakenly caused the problem by modifying the Joomla ACL (Access Control List). This is a common problem and we really think that Joomla must implement measures to automatically mitigate this problem (one shouldn’t be allowed to intentionally or unintentionally corrupt the Joomla ACL while working from within Joomla).

If you are seeing an empty backend after logging in to your Joomla website, then most likely your #__viewlevels table is corrupt. Follow the above instructions to fix it and you should be all set (make sure you backup your database first). If you think the above is a bit over your head, or if you’re just a bit afraid of doing it yourself, then please contact us. We are always ready, we are always happy to help, and our fees are super duper affordable.

Why Is It that Joomla Doesn’t Care About the Category Alias in Article URLs

There is a confusing “feature” in Joomla, and it is that the CMS, unintentionally, can have multiple links pointing to the same article. For example, if you go to the following URLs http://www.[your-joomla-website]/category-1/12-test-article.html and http://www.[your-joomla-website]/category-2/12-test-article.html then both URLs will work despite the fact that they point to the same article, and despite the fact that Test Article belongs to Category 1, and not Category 2.

And why is that a problem?

It is a problem, and a big problem, for that matter, because it will negatively affect rankings in search engines, especially Google rankings (Google will think that the website administrator is trying to spam its database with multiple listings for the same article). Imagine if you have 100 categories and 500 articles on your website. A worst case scenario will mean that you will end up with 50,000 different URLs instead of 500 – which will certainly dilute the strength of each and every URL. From an SEO perspective, this is a nightmare…

So, what causes this issue?

The issue is caused by the parse function which is located in the router.php file (which, in turn, is located under the components/com_content folder), which only takes into consideration the last part of the link for content items when displaying them. So, not only you can have a different category alias in front of the article alias, you can literally have anything! So even the following will work: http://www.[your-joomla-website]/abcdefg1234567/12-test-article.html (try it!), when abcdefg1234567 is not even a category alias (by category alias, we mean a menu item of type Category Blog with an alias that is the same as the alias of the category it is pointing to). What’s even worse is that the home menu item ID (which is the default one) will apply to these weird links, and so all the homepage modules will be assigned to them…

So, how can this problem be solved?

The best solution to this problem is through the SEF Advanced Link, which is a secret feature in Joomla (its mystery is only exceeded by its power, just like the Continuum Transfunctioner). All you need to do is to set it to 1, and we explained how to do that here. Now, if you’re wondering, the reason why doing this will fix the problem is because when this is set, the article ID will no longer appear in the URL, and so Joomla will have no option but to check all the parts of a URL in order to know which article it should display, as the article alias by itself is not sufficient to correctly locate an article (in Joomla, article aliases do not have to be unique).

The downside of the fix above is that it will alter the URL structure of the Joomla website (it will remove the ID of the article from the URL), which can result in a problem bigger than the original problem especially if you have too many articles on your website. So, what you will need to do is develop as simple script (and maybe place it in the defines.php file under the root directory of your website) which will check if the current URL has an ID in front of the alias, and, in case it finds one, then it will issue a 301 REDIRECT to redirect to the URL without that ID.

We have tried out best to make this post as easy as possible, but we reckon that it’s still somehow complex to implement, especially with the last part (redirecting from the old links to the new links). So, if you need help with the implementation, then please contact us. Our work is professional and efficient, our rates are affordable, and we are the most reliable Joomla developers on planet Earth!

A Very Scary VirtueMart Exploit, or Something Else?

A new client called us a couple of hours ago and told us that he had a problem on the VirtueMart store on his Joomla website. He said that early in the morning a client called him and asked him about the status of his one week old order which he paid for through PayPal. Our client was perplexed, because he doesn’t have PayPal as payment method (he sells e-cigarettes and accessories on his online store, which means that he can’t transact through PayPal [PayPal doesn’t allow merchants selling e-cigarettes to transact through their system]). So, our client searched through the VirtueMart orders, and found an order matching that of his client (our client doesn’t use VirtueMart to see the orders, as he uses an application called ShipWorks to do that [which only lists orders that transact successfully through Authorize.net], that’s why he didn’t see that order before). To make a not-so-long story concise, the website made the sale, but someone else took the money.

We immediately investigated the issue, and we noticed the following:

  • The website was using the latest version of Joomla.
  • The website was using the latest version of VirtueMart.

  • The PayPal VirtueMart plugin was installed on the website, and it was configured to accept money to someone with a French email address (the email ended with .fr). The fraudulent orders totaled a tad over $200.

  • There was a weird super user with an even weirder registration date. That user never logged in to the website (according to Joomla anyway).

In order to address the problem, we did the following:

  • We uninstalled the PayPal plugin (we took note of the fraudulent email account first and forwarded it to the client).
  • We removed the weird user from Joomla.

  • We changed the password for the following: the Joomla super user, the cPanel account, the FTP account, and the MySQL database powering the Joomla website.

  • We scanned the website for viruses/backdoors (there were none).

  • We installed our own firewall on the Joomla website.

So, what caused this problem?

We don’t know (we weren’t commissioned to do so), but we have the following theories in our mind:

  • The client did not update the Joomla website to 3.6.4 immediately, so his website suffered from the elevated permissions exploit for a while before updating it, and during that time, someone registered the weird Joomla super user, and used it to install and configure the PayPal plugin. This is a weak theory, however, because the weird Joomla account was never used.
  • Someone knew the super user password, and used it to install the PayPal plugin and configure it. This is a plausible theory because the super user password was super easy.

  • An attacker used a hidden exploit in VirtueMart and installed that plugin. This is improbably but not impossible, and if it is true, then many e-commerce shops are at risk.

We can’t be really sure of the root cause of this problem (we were not commissioned to investigate further), but we’re leaning towards the simple theory, which is the second theory, where someone knew the super user password and used it to install and configure the VM PayPal plugin.

If you’re running a VirtueMart powered online store, then we suggest you verify your payment settings; the prospects of someone changing the payment information on your website to his own are really terrifying. If you found something fishy on your website and you need help, then please contact us. We will clean your website, we will protect it, and we won’t charge you much.

Are You Unable to Edit Your Joomla Articles? Read This!

Note: This post is extremely advanced. Only try to implement the below if you’re a solid developer and after backing up the Joomla website (database + filesystem).

We had a weird case this morning. A client called us and told us that her staff were able to edit certain articles only once every day – in other words, they edit the article the first time, they save it, and then they can’t do edit it again until the next day.

That was very weird, we thought. We haven’t seen something remotely similar in a decade of Joomla consulting. So, we visited the website, and we edited one of those cursed articles, and then we saved it and closed it. To our surprise, the article was no longer editable (the article title was no longer a link).

We knew this whole thing had something to do with the #__assets table, but we just didn’t know where to look. However, after examining the issue further we noticed that the all of these articles belonged to the same category, which was the Uncategorized category.

So We looked up the entry of the Uncategorized category in the #__assets table, and we couldn’t find one, we were, however able to find an entry in the #__assets table matching the value of the asset_id field of that category, and that entry was that of a module (its name was com_modules.module.334). Clearly, that was wrong…

After some testing, we discovered what was happening, when an article belonging to the Uncategorized category was edited and saved, a wrong entry for that article was created in the #__assets table. Why is it a wrong entry? Well, because the parent of that entry is not the right parent (it is a module instead of the Uncategorized category). Obviously, fixing the problem consisted of recreating an entry in the #__assets table for the Uncategorized category. Here’s how we did it:

  • We created a new category, we called it “Test Category”. We made sure that it had the same parent category as that of the Uncategorized category (it actually had no parent).
  • We went to the #__assets table, and then we searched for the entry corresponding to “Test Category”.

  • We changed the value of the name field for that entry in the #__assets table to be com_content.category.33 (where 33 is the ID of the Uncategorized category in the #__categories table), we then changed the title field in the #__assets table to be Uncategorized.

  • We changed the value of the asset_id field of the problematic (Uncategorized) category to that of the ID of the row that we have just changed in the #__assets table.

  • We deleted the Test Category from the #__categories table.

  • We ran the following query on the #__assets table…

    DELETE FROM #__assets WHERE name = 'com_content.article.nnn';

    where nnn is the article ID that we were not able to edit. Doing this removed the bad asset information associated with that article.

  • That’s it. The problem was fixed.

So, what caused the problem in the first place?

We don’t know what caused this data corruption in the #__assets table, but we think it was one of the following:

  • A previous migration that wasn’t 100% clean.
  • A poor extension that corrupted the #__assets table.

  • An incorrect manual edit to the #__assets table.

Why were they able to edit articles the next day?

Well, because the website had a cron job to optimize the #__assets table at midnight of each day. The optimization consisted of pruning all the articles entries in the #__assets table as per our guide here. Once the cron ran, all the bad references in the #__assets table were removed, and so editing articles belonging to the Uncategorized category was possible again.

If you can’t edit articles on your website, then check your #__assets table, and implement the guide above. If you have problems with the implementation, or if you’re too scared to do it, then please contact us. We will do the work for you swiftly, professionally, and for very little money.

“Application Instantiation Error: Table ‘db.#__session’ doesn’t exist” Error in Joomla: 5 Possible Reasons

Note: You must replace #__ in the post below with your database alias which is the value of the variable $dbprefix that is defined in the configuration.php file of your Joomla website.

Another note: Always backup your Joomla database before making any modifications to it.

A not so uncommon fatal error on Joomla websites is the following error:

Application Instantiation Error: Table ‘db.#__session’ doesn’t exist

What the above error essentially means that Joomla is unable to read/write to the critical session table. This can be caused by one of the following (the below are listed in order of occurrences):

  1. The session table is corrupted: The most common reason of the Table ‘db.#__session’ doesn’t exist error is corruption in the #__session table, this is especially the case when the table is using the MyISAM database storage engine (which is infamous for causing table corruption). If this is the case, then you should repair the table, which can be done by issuing the following query in phpMyAdmin:

    REPAIR TABLE `#__session`

    The above query should be sufficient to repair the table. If it doesn’t work, then you may need to reconstruct the table from scratch, which is described in the following section.

    Note: We recommend switching the database engine of the #__session table to InnoDB or to MEMORY as MyISAM is a very flimsy database engine.

  2. The session table was deleted: One some occasions, the cause of the problem is a deleted #__session table. In this case, you will need to reconstruct the table. Reconstructing the table also works if the table is corrupted but cannot be repaired using the REPAIR syntax. Reconstruction of the #__session table is a safe process because the information in the #__session table is meant to be volatile and can be safely deleted at anytime.

    To reconstruct the table, you will need to run the following queries in phpMyAdmin (the following is compatible with Joomla 3.x):

    DROP TABLE `#__session`;
    CREATE TABLE IF NOT EXISTS `#__session` (
      `session_id` varchar(191) NOT NULL DEFAULT '',
      `client_id` tinyint(3) unsigned NOT NULL DEFAULT 0,
      `guest` tinyint(4) unsigned DEFAULT 1,
      `time` varchar(14) DEFAULT '',
      `data` mediumtext,
      `userid` int(11) DEFAULT 0,
      `username` varchar(150) DEFAULT '',
      PRIMARY KEY (`session_id`),
      KEY `userid` (`userid`),
      KEY `time` (`time`)
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 DEFAULT COLLATE=utf8mb4_unicode_ci;

    The above code should drop your #__session table and then reconstruct it.

  3. The database user doesn’t have full read/write control over the session table: A not so uncommon cause of the problem is a user which doesn’t have full privileges over the #__session table. This is likely to happen when the Joomla database is managed at a granular level, and not through a mainstream system (like WHM). Fixing the problem is usually done by granting full read/write permissions on the #__session table (this is typically done by a system administrator).

  4. The database is empty: Last week, we had the Table ‘db.#__session’ doesn’t exist problem on a website, and it turned out that the database was empty. It didn’t have a single table, not one. The database did exist and the connection parameters were correct, but it was empty (if the database didn’t exist, or if we had the wrong database credentials in the configuration.php file, then we would have seen the Application Instantiation Error fatal error instead). We fixed this problem by reverting to an old backup. The backup didn’t contain the most up-to-date data, but it was much better than nothing.

  5. The configuration file is pointing to a different database: On some instances, we have seen a configuration.php file where the database parameters are pointing to an existing database, but not to the Joomla database. This is not common but it can happen, and it is usually caused by a human error: for example, the system administrator copied the content of the Joomla database to another database, emptied the original database (and used it for a different purpose), but did not update the parameters in the configuration.php file to point to the new database. Fixing the problem simply consists of updating the database parameters in the configuration.php file to point to the actual database.

We hope that you found our post useful and that it helped you solve your problem. If it didn’t, then you can always contact us. We are always happy to serve, our prices are super cheap, and we really love to have you as a client and a friend!

Using Apache’s Substitute Module to Remove the Joomla Meta Generator Tag

At itoctopus, we love doing things in different ways, this helps us broaden our horizon and it also allows us to provide several alternative solutions for our readers, who may prefer one method over another.

In a previous post, we have discussed, in details, 2 methods for hiding the Joomla version. The first one consisted of modifying a core file (which we don’t usually recommend), and the second one consisted of just modifying the template file.

In this post, we will explain a third method to do that, and it is through the .htaccess file. Yessiree Bob, you heard that right, the only change that you will need to make in order to remove the Joomla meta generator tag is through the .htaccess file.

So, how is it done?

Well, there is this little Apache module called mod_substitute (note: Apache modules are not the same as Joomla modules – Apache modules, if you really want to know more about them, work more or less in the same way as Joomla plugins), which (unsurprisingly) allows the substitution of one pattern with another in the served HTML page. So, to replace the Joomla meta generator tag on the website, all we need to do is to add the following lines to the beginning of the .htaccess file:

<IfModule mod_substitute.c>
	AddOutputFilterByType SUBSTITUTE text/html
	Substitute "s|<meta name=\"generator\" content=\"Joomla! - Open Source Content Management\" />||i"
</IfModule>

That’s it!

What if it doesn’t work?

If it doesn’t work, then most likely you don’t have the mod_substitute Apache module installed and/or enabled on your server. This can be quickly verified by using one of the following methods:

  • Issue the following command in the Linux shell:

    httpd -M | grep 'substitute'

    If you get an empty result, then you don’t have mod_substitute installed.

  • Remove the first line and the last line from the .htaccess code above (only if you are running on a development server), and your website should display a 500 error. Additionally, you will see the following entry in your Apache error log (/usr/local/apache/logs/error_log in a WHM/cPanel instance): Unknown filter provider SUBSTITUTE.

Once you confirm that you don’t have mod_substitute installed, then you can talk to your system administrator about it and he should be able to sort it out for you.

Are there any other uses of mod_substitute?

There are. In fact, it is mostly used to append some JavaScript tracking code to the end of the served page. This is very practical for a system administrator who’s asked to add some tracking code to a Joomla website, but who doesn’t know how to work with it (with the Joomla website).

Are there any disadvantages of using mod_substitute?

Aside from a slight-to-negligible performance hit, there are no technical disadvantages for using mod_substitute. There are, however, some reservations. For example, when you use mod_substitute, you are transferring some of the logic on your website to the .htaccess file, which is not good. You will also make the process of maintaining your website confusing. If you want our opinion, then while we think that mod_substitute is a wow feature (there are only a few people out there who know that you can replace the HTML on a page using .htaccess), we think it should be used only when it’s really needed, and not because of its mystique.

If you think that this whole thing is a bit over your head and you need help with the implementation, then we’re here to help! Just contact us and we’ll implement the above for you swiftly, professionally, and for very little money!

“Blocked loading mixed active content” when Using Joomla’s SEF in HTTPS Mode

Yesterday noon, a new client contacted us and told us that he was having problem when using HTTPS on his Joomla website (the website in question was using the latest Joomla version, which is 3.6.4). He said many images did not appear, and it seemed as if no CSS was applied to the website whatsoever. So, we loaded his website in our browser (we were using FireFox), and then we checked the console log of the browser, which showed the following errors:

Blocked loading mixed active content “http://www.[our-client-joomla-website].com/templates/[our-client-joomla-template]/css/bootstrap.css”
Blocked loading mixed active content “http://www.[our-client-joomla-website].com/templates/[our-client-joomla-template]/css/default.css”
Blocked loading mixed active content “http://www.[our-client-joomla-website].com/templates/[our-client-joomla-template]/css/template.css”
Blocked loading mixed active content “http://www.[our-client-joomla-website].com/templates/[our-client-joomla-template]/css/touch.gallery.css”
Blocked loading mixed active content “http://www.[our-client-joomla-website].com/templates/[our-client-joomla-template]/css/responsive.css”

Looking at the above errors (there were many other similar errors), we immediately knew what the problem was: the website was trying to load HTTP content when in HTTPS mode. The browser, in order to protect the end user, blocked all non HTTPS content.

What was interesting about this whole thing was that the problem disappeared as soon as we switched off Joomla’s SEF plugin. In other words, the problem was technically caused by Joomla’s SEF plugin (the System – SEF plugin).

Naturally, the next step was to examine the HTML code, and so we looked at the HTML code very thoroughly and very carefully, and we were surprised when we saw that all the CSS/JS/image references were relative, and not absolute, meaning we shouldn’t have a problem with them, but we also noticed another thing, it was this line in the head tag:

<base href="http://www.[our-client-joomla-website].com/" />

Aha! So the base href was set to HTTP, and that’s why all the relative links were also set to HTTP. So, our task was to change the http in the above line to https. In other words, we needed to change the above line to:

<base href="https://www.[our-client-joomla-website].com/" />

So, how did we fix it?

Fixing the problem was very easy. All we needed to do was to open the index.php file in the templates/[our-client-joomla-template] folder and add the following code immediately after <?php defined( ‘_JEXEC’ ) or die( ‘Restricted access’ );?>:

<?php $document = JFactory::getDocument();
$document->base = str_replace('http://', 'https', $document->base);?>

That was it! When we refreshed the page after adding the above code, all the Blocked loading mixed active content errors were gone!

But why did the Joomla SEF plugin cause this problem?

While the enabling the System – SEF plugin did result in the problem, we don’t think it was the root cause. We think that the root cause lied elsewhere (no – it wasn’t the $live_site in the configuration.php file). Unfortunately, we were not commissioned to find out what the root cause was, as the client was happy with the fix that we provided (which was pretty solid anyway).

If you’re having the same problem on your Joomla website, then try our solution above. If it didn’t work for you, or if you’re having problems with the implementation, then please contact us. Our work is quick, our results are professional, and you won’t have to pay an arm and a leg (at least not both) to afford us.

“The most recent request was denied because it contained an invalid security token. Please refresh the page and try again.” Error when Updating a Joomla Website

While trying to update the Joomla website of a regular client of ours, we were faced with the following error:

“The most recent request was denied because it contained an invalid security token. Please refresh the page and try again.”

If you are an avid reader of our blog, then you will know that the above error is one of the most dreaded Joomla errors out there. While we have devised a method to get rid of the invalid token problem, we admit that our solution is really about hiding the actual issue, rather than resolving it. Additionally, it was the first time that we have seen this error when trying to update a Joomla website.

Further examination of the update process, we noticed that Akeeba intercepts the update process to make a full backup of the website before the actual update. So we thought, what if the problem had to do with Akeeba?

Knowing that Joomla and Akeeba work closely together, we were confident that if the problem was caused by Akeeba, then it must have been resolved in the latest Akeeba version. So, we updated Akeeba from 4.7.4 to 5.2.4, which is the latest version at the current time, and we tried to update the website again, and this time, it worked. There was no Invalid security token error or any other error for that matter – the update worked super smoothly.

But why did the problem occur with the previous version of Akeeba?

We don’t know – but most likely it was a compatibility issue with Joomla’s updated session management engine. Luckily, the Akeeba people were proactive and fixed the problem immediately.

So, if you’re seeing the “Invalid security token” error on your Joomla website when you’re trying to update it, and if you have Akeeba backup installed, then make sure you update Akeeba backup to the latest version as that might solve your problem. If it doesn’t, then please contact us. We will fix the problem for you quickly, affordably, and professionally.

JCE Editor Crashing on a Specific Joomla Article

We think very highly of JCE and we have huge respect for its developers. We recommend it to our clients and we always install it on the Joomla websites that we fully manage. With the exception of very few quirks, our experience (and that of our clients) is very smooth with JCE.

The very few quirks that JCE has, however, are very irritating, even more irritating than having mayonnaise in a burger, and you know we don’t like mayonnaise, don’t you? (yes – this line is inspired by Jimmy the Tulip.) On the bright side, however, all the quirks that we have experienced with JCE were addressed by modifying its settings (such as this one), which technically doesn’t make them quirks.

But, this morning, a very important client of ours emailed us and told us that they were not able to save a specific Joomla article. They said that the browser was hanging for that specific article and, as such, they weren’t able to make any modifications to the article.

We immediately checked the Joomla article, and we noticed that it had many images, and so we thought that the browser was crashing because of this. So, we went to phpMyAdmin, and we removed all references to the images from the article, and then we tried loading the article from the Joomla backend. Still, the same problem: the editor crashed.

We then thought, what if there are settings in JCE that are causing this crash? What if, for example, JCE was set to validate the HTML and the HTML validation is crashing? So, and while in the Joomla backend, we went to Components -> JCE Editor -> Global Configuration, and, to our joy and rejoice, we noticed that HTML validation was turned on! So we set the Validate HTML field under Cleanup & Output to No. We tried the page again but, unfortunately, it kept crashing. We then set the Compress Javascript, the Compress CSS, and the Compress with Gzip fields under Compression Options to “No”, but that didn’t help either.

At this point, we knew that the problem was with the JavaScript code of the JCE editor, and this made us somehow desperate. Why? Because only a few programmers on planet Earth would love to work on the JavaScript code of the JCE editor, and we’re not among those programmers.

But then we thought, why is the problem happening only on this page and not on other pages? Maybe there is something “special” on this page that is causing the JavaScript to crash. So we started the process of elimination, in other words, we removed, bit by bit, HTML code from the article in phpMyAdmin until the problem was no more, and we finally were able to find the root cause of the problem, it was this line:

<a href='http://www.[ourclientjoomlawebsite].com/page.html'>Click here to continue &nbsp; &gt;&gt; </a>

But what is “special” in the above line? Well, it was this &nbsp;&gt;&gt; bit, for some reason, JCE editor did not love this combination in the anchor tag. Removing the above did fix the problem.

We will report this issue to the JCE development team on Twitter. Meanwhile, if the JCE editor on the backend of your Joomla website is crashing for a specific article, then make sure you 1) update your JCE editor to the latest version, and 2) try our process of elimination above. If none of the above works, then please contact us. We are always happy and ready to help, we love having new clients, and we our fees are super affordable.

“Fatal error: Class ‘JFeedItem’ not found” When Overriding the Joomla Core

Note: This is an advanced post which mentions an unstable extension at the time of writing (November 2016). We suggest you avoid overriding the Joomla core if you’re not comfortable with your programming skills. If you need help with overriding the Joomla core, then you can always contact us!

Until very recently, we used to overwrite the core mainly to resolve performance issues. But, last week, we have discovered a not-so-new extension, an extension called Joomla Override, which allows developers to override the Joomla core. So, we decided to give this extension a chance… But, before delving into the details of our experience, let us explain the difference between overwrite and override.

In case you don’t know already, the basic difference between overwriting and overriding is that in the former, you open a Joomla core file, you make the changes, and then you upload it back. In the latter, you copy the Joomla core file (that you want to override) to a different location and you make the changes there. Obviously, overriding has a huge advantage over overwriting, and it’s that the changes don’t get wiped out with a Joomla update. Another major advantage is that all the overrides are in one location – instead of having them spread over the many folders of the Joomla website.

So, how did our experience go with using the Joomla Override plugin?

In all fairness, the experience didn’t go very well, because the moment we tried to override the Joomla feed of the homepage, we saw the following error on the homepage of the Joomla website:

Fatal error: Class ‘JFeedItem’ not found in templates/[our-client-joomla-template]/code/com_content/views/featured/view.feed.php on line 66

As mentioned above, the error was on the homepage of the Joomla website, and not on the feed page, which we thought was odd. Why would overriding the feed file affect the homepage? So we thought, it might be that we also need to override the view.html.php file (and not just the view.feed.php file), and so we copied the view.html.php file from the components/com_content/views/featured folder to the templates/[our-client-joomla-template]/code/com_content/views/featured folder.

This time, when we refreshed the page, we saw the following (different) error:

Fatal error: Cannot redeclare class ContentViewFeatured in templates/[our-client-joomla-template]/code/com_content/views/featured/view.html.php on line 0

After a long and tedious research of the above problem, we discovered that it was caused by 2 issues:

  • Joomla has the exact same class name in both the view.html.php and the view.feed.php files (for both the category and the featured views in the com_content component). We consider this to be a Joomla bug as Joomla shouldn’t have the same class name in 2 different files.
  • The Joomla Override plugin doesn’t care about the current format, and loads both the view.feed.php and the view.html.php despite the fact that they are almost never needed at the same time.

So, how did we fix the problem?

We fixed the problem the following way:

  • We opened the file component.php located under plugins/system/joomlaoverride/helper
  • Just before the following line:

    if ($params->get('extendDefault',0))

    We added the following code:

    $arrRealFileName = explode('/', $filePath);
    $realFileName = $arrRealFileName[count($arrRealFileName) -1];
    $format = JFactory::getApplication()->input->get('format','html');
    if (stripos ($realFileName , $format) === FALSE)
    	continue;

  • We saved the file and then uploaded it back. We then refreshed the homepage, and the problem was fixed!

Did things go smoothly afterwards?

Well, not really. We discovered that there were some important core files that we were not able to override. For example, we wanted to override the file cms.php which is located under the libraries/cms/application/ folder, mainly so that we can disable session activity for visitors , but we weren’t able to. Even if we were able to, the override was useless, simply because session activities take place before any plugin is loaded. So, we had to overwrite the cms.php file instead of overriding it.

In total, we had to overwrite about 4 files for a large Joomla website, which wasn’t that bad, but wasn’t that good either. In addition, we were faced by several problems when we used this extension (including the one in this post), all of them were hard to fix. Because of that, we can’t recommend using the Joomla Override plugin, in fact, we recommend against using it, until it is stable enough.

If you want to override/overwrite some core files on your Joomla website, then please contact us. We will amaze you with our professionalism, cleanliness, and our super affordable fees!

How to Create a Custom HTML Module in Joomla’s Backend

Sometimes, large companies have some text that they want to display somewhere on the backend of their website for their staff. The text might consist of editing instructions, a legal disclaimer, or a general announcement. Now, on the frontend, it is very easy to display such a text by using a Custom HTML module (nowadays, it is called Custom module, without the HTML), but how about the backend?

Well, luckily, the backend also has a Custom module that can be used the same exact way as a site Custom module. Here’s how you can use it…

  • Login to the backend of your Joomla website.
  • Click on Extensions -> Modules.

  • On the top left (just below the green New button), choose Administrator from the dropdown that displays Site.

  • Click on the green New button on the top left.

  • Choose Custom from the Select a Module Type page.

That’s it! Now the only thing that you need to worry about is the position of the module – which should be very easy to figure out once you click on the Position dropdown on the right. Note that you must know which administrator template you are using in order to choose the correct module position. If you don’t know which template you’re using, then just click on Extensions -> Templates, and then, in the dropdown that has Site in it, choose Administrator. The administrator template that you’re using has a yellow (or is it amber?) star next to it.

Is the administrator Custom module the exact same as the site Custom module?

While both modules are very similar, they are not the same: the administrator Custom module and the site Custom module use different files (they do not share the same files), and the files have some slightly different code.

Can the module’s layout be overridden?

Well of course. All you need to do is to create to copy the default.php file from under the administrator/modules/mod_custom/tmpl folder to the administrator/templates/[your-joomla-administrator-template]/html/mod_custom folder (you will need to create the mod_custom folder if it doesn’t exist).

We hope that you found our little guide above helpful in creating Custom HTML modules in the backend of your Joomla website. If you need help with the implementation, then please contact us. We’ll do the work for you swiftly, professionally, and for very little money!

“Unknown SSL protocol error” on Joomla’s VirtueMart Checkout

A client of ours emailed us yesterday and told us that clients weren’t able to checkout on his VirtueMart store. He told us that they were seeing the following error:

Unknown SSL protocol error in connection to www.eprocessingnetwork.Com:443

We did a quick research on the issue and it looked like it was caused by an update to the SSL certificate on the destination (e.g. on the eProcessingNetwork website), and so we tried adding the following code in the authorizenet.php file (just immediately after curl_setopt($curl_request, CURLOPT_RETURNTRANSFER, 1);), which is located under the plugins/vmpayment/authorizenet folder of the Joomla website:

curl_setopt($curl_request, CURLOPT_SSLVERSION, 3);

We tried the transaction again and we had the same problem. And so we flipped the 3 in the above line to 2:

curl_setopt($curl_request, CURLOPT_SSLVERSION, 2);

Still, we had the same error.

And then we thought, what if the OpenSSL library on the source server (e.g. on the server hosting the Joomla website) was outdated. So, we logged in to the WHM account, and we saw a big yellowing warning on the top stating the following:

Red Hat® will deprecate all CentOS 5 systems on March 31, 2017. cPanel, Inc. and Red Hat will no longer provide your operating systems with updates or security fixes. You must migrate your server to a CentOS 7 server with the Service Migration Tool to ensure that your server remains up to date. You must contact your hosting provider for a destination server.

Aha! So the server was using a very old version of CentOS (which is CentOS 5), hence the outdated OpenSSL library. That meant that the only solution to the problem was to update the OS on the server, and so we initiated the update process with the hosting company, and they took care of the rest.

We started testing the website immediately when the update to CentOS 7 was done, and, unsurprisingly, payment processing worked again! The problem was fixed!

We hope that you found our little post helpful and that it helped you fix the checkout problem on your Joomla website. If it didn’t, then please contact us. We’ll fix the problem for you quickly, professionally, and for very little money!

A Downloadable List of the Top 500 User Agent Strings on a High Traffic Joomla Website

This morning, we thought we had a little time to do something fun, and so we created a command to generate a list of all user agent strings (or signatures) on a very high traffic Joomla website that we maintain. “Why is that?”, we hear you ask… Well, because 1) we were curious about which user agent signature is the most common on the Internet, and 2) we had some security concerns because of the 500 HTTP Errors that we found a few days ago on that particular website and which were caused by a fake user agent signature.

So, what is the most common user agent string as of October 2016?

Well, if you really are interested, then the most common user agent string (as of October 2016) is (drum rolls, please):

Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko

In case you’re wondering which browser the above user agent string belongs to, then it’s FireFox (we love FireFox, although we have to say it’s becoming more memory hungry with each iteration).

And what are the top 500 user agent strings?

A list containing the top 500 user agents strings can be downloaded here. The original raw list contained 53530 unique user agent strings (that’s unique, in case you didn’t notice the underline), and we generated it by issuing the following command in the Linux shell (note that the below will just return the top 500 user agent strings, if you care about the whole list, then you should delete this part | sed -n 1,500p from the below string):

awk -F\" '{print $6}' [ourclientjoomlawebsite].com-Oct-2016 | sort | uniq -c | sort -k1nr | awk '!($1="")' | sed -n 1,500p > user-agents.txt

We know that you’re curious about the above set of commands, so let us dissect them for you:

  • The first command…

    awk -F\" '{print $6}' [ourclientjoomlawebsite].com-Oct-2016

    …grabs the 6th column from the deflated Apache log file. The 6th column is typically the user agent string.

  • The second command…

    | sort

    …uses the list returned by the first command as input (which is a raw list of all the user agent strings), and sorts it by user agent (this is necessary for the subsequent command).

  • The third command…

    | uniq -c

    …creates a unique list of all the user agent strings (out of the result of the second command), where each line consists of a user agent string, preceded with the number of times that that user agent string occurred in the logs.

  • The fourth command…

    | sort -k1nr

    …sorts the list returned by the previous command by number of occurrences of user agent strings descending.

  • The fifth command…

    | awk '!($1="")'

    …removes the first column (which contains the number of occurrences of each user agent string).

  • The sixth command…

    | sed -n 1,500p

    …returns the first 500 lines (e.g. the top 500 user agent strings, ordered by number of occurrences, descending).

  • The seventh command…

    > user-agents.txt

    …redirects the output to the user-agents.txt file.

As you can see, it’s not that hard once it is explained! Nevertheless, one has to admit it’s a bit hard, however, it clearly highlights the power of shell commands in Linux (if you’re a PHP developer, then imagine how much code would be needed to do all the above in PHP).

If you decide to run the above set of commands, then please keep in mind that it will take about 10-15 minutes to process a 10 GB log file even when using a relatively powerful server with an SSD drive.

A couple of notes about the generated file…

The Apache log that we have used to generate the list is that of a very professional website, which main visitors work for professional companies and governmental agencies. Additionally, the website in question has the absolute majority of the known spam bots blocked (which explains the absence of many mainstream spam bots in the list).

We hope that you found this post fun and useful. If you need help generating an even more interesting set of information from the Apache logs of your Joomla website, then please contact us. We are particularly fond of Linux (and awk), we love our job, we love our clients, and we won’t charge you much!

How to Quickly Know the Version of any Joomla Website

Let’s say you want, for one reason or another (hopefully a good reason), to know the version of a Joomla website that you don’t own. You don’t have FTP/sFTP access to the site’s filesystem, you don’t have access to the backend, and you don’t know the owner. So, what do you do?

Well, for the absolute majority of Joomla websites, there is at least one file that you can check that will tell you the exact version the Joomla website is running.

For Joomla websites >= 1.6.0

Joomla websites, ever since version 1.6.0, have a very easy method that reveals their exact version (which may or may not be a good thing), all you need to do is to access the following URL:

http://www.[thejoomlawebsite].com/administrator/manifests/files/joomla.xml

The above URL will display an XML file containing the site’s version in the version XML element.

Yes – it’s scarily easy, huh! It also applies to all versions of Joomla from 1.6.0 until 3.6.3 (including, of course, all the versions in the 2.5.x line), which is excellent!

For Joomla websites >= 1.5.0 and <= 1.5.26

Joomla websites in the 1.5.x line do not have the joomla.xml file, but there is another link that one can check, which is this one:

http://www.[thejoomlawebsite].com/language/en-GB/en-GB.xml

You can clearly see, in the XML version element, the version of the Joomla website in question. However, the problem here is that this method only reveals the major version, and not the minor version. The minor version is typically guessed by checking some core JavaScript files under the media/system/js folder.

For Joomla websites < 1.5.0

Before you say anything, yes, there are still some Joomla websites on the Internet using the 1.0.x versions (in fact, we worked on one back on Sunday). In order to know if a Joomla website is version 1.0.x, then all one needs to do is to check for the existence of a custom.xml file by visiting the following URL:

http://[thejoomlawebsite].com/modules/custom.xml

Similarly to the Joomla 1.5.x line, the Joomla 1.0.x line does not have a straightforward method to accurately determine the minor version of the Joomla website (for example, if it’s using version 1.0.0 or version 1.0.15). However, JavaScript files in the includes/js folder typically reveal information (such as file revisions) that will help determine the minor version of the Joomla website.

In short…

  • If the link http://[thejoomlawebsite].com/modules/custom.xml works: This is a Joomla 1.0.x website.
  • If the link http://www.[thejoomlawebsite].com/administrator/manifests/files/joomla.xml works: This is a Joomla website with version >= 1.6.0, and the exact version number is stored in the version XML element of the file.

  • If none of the above links works, then the Joomla website is a 1.5.x website.

Again, the above assumes that you don’t have filesystem access to the website. If you do have access to the filesystem, then you can check the file version.php which tells you the exact version of the Joomla website. Unfortunately, the version.php file is not present in a standard location. Luckily, however, you have itoctopus on your side to let you know of its exact location for each Joomla version:

  • For Joomla 1.0.x websites: The file version.php is located under the includes folder.
  • For Joomla 1.5.x and Joomla 1.6.x: The file version.php is located under the libraries/joomla folder.

  • For the short lived Joomla 1.7.x: The file version.php is (oddly) located under the includes folder (which is similar to 1.0.x websites).

  • For Joomla 2.5.x and Joomla 3.x: The file version.php is located under the libraries/cms/version folder.

We hope that you found this post useful. If you didn’t, or if you have just discovered (after following the above guide) that you are running a very old version of Joomla and you have decided that you want to update it, then please contact us. We are always here for you, we do the job right, and our fees are super duper affordable!

500 HTTP Errors – Revealing Vulnerabilities on Your Joomla Website

One of the biggest mistakes that system administrators make when analyzing the Apache server logs, is that they ignore many of the “500” HTTP errors (also known as Internal Server Errors). They think that these fatal errors – if they are not happening on legitimate pages, then they are not worth investigating. Little do they know that there is usually much more beneath the surface. Let us give you an example…

This morning, as part of a routine security check on a Joomla website, we ran the following grep command to get a list of all the pages that generate 500 errors:

grep ' 500 ' ourclientjoomlawebsite.com-Oct-2016 > 500-errors.txt

(Note: The file ourclientjoomlawebsite.com-Oct-2016 is the Apache access log file for the website ourclientjoomlawebsite.com).

When we examined the 500-errors.txt file, we saw the following fatal error, many times:

[ip] - - [16/Oct/2016:14:14:09 -0400] "HEAD /component/mailto/?tmpl=component&template=template-name&link=http://www.ourclientjoomlawebsite.com/category-1/page-1.html HTTP/1.1" 500 - "-" "Mozilla/5.0 (compatible; um-LN/1.0; mailto: randomemail@[anotherdomain.com])"

Now, if you just copy and paste the above link to a browser (of course, you will have to append the domain name in front of it), then you will notice that the link works (it will show the “Send this Article to a Friend” form), but there’s more to it…

The user agent, which is the root of a substantial amount of evil in the online world, has an email in it. Obviously, such a user agent is fake, and has been specifically tampered with to crash the Joomla website, and it did do its job!

The thing is, not a single request on a Joomla website (or any other website, for that matter) should crash it. A solid website should handle any request gracefully, and should never crash, because a crash is a huge positive sign for a blind SQL injection (which may lead the way for an actual exploit).

Naturally, we fixed the problem by 1) addressing the issue in the mailto extension, and 2) by cleaning up the $_SERVER[‘HTTP_USER_AGENT’]. But, we thought, what would have happened if we didn’t examine the server logs this morning? Would it have been possible for the attacker to exploit the website? The latter thought sent chills through our spines, because the website in question was for a very important client of ours.

So, as a system administrator on a Joomla website, it is up to you to examine the server logs for “500” errors, and it is up to you to immediately report any of these errors to your developer(s). If you don’t have any developers to handle these errors, or if you need help performing the log analysis on your website, then please contact us. We are always happy to serve and our fees are super affordable!

A High PHP Memory Limit in Joomla = An Invitation for DoS Attacks

Occasionally, Joomla administrators face the infamous allowed memory size error which forces them to increase the memory_limit value in the global php.ini or in a local .user.ini file by adding the following line:

memory_limit=256M

The above code will increase the memory limit to 256 megabytes, which is more than ample for any Joomla website out there. Now, the thing is, the absolute majority of the times, Joomla administrators increase the memory limit to allow an occasional usage of a certain Joomla functionality, such as updating the Joomla website or installing an extension. Once the update is done or the extension is installed, then there is no need for the high PHP memory limit anymore.

Unfortunately, most Joomla administrators do not revert back the changes they did to the PHP memory limit – mainly because they think it’s better this way (it did fix the problem – didn’t it?). But it’s not – in fact, a high memory limit makes a DoS (Denial of Service) attack easier and much more dangerous. Let us explain…

The PHP memory limit defines how much memory each PHP page is allowed to consume – so if, for instance, you have a memory limit of 256 megabytes, then, assuming a worst case scenario, merely 8 simultaneous page views can take up 2 GB of your server’s RAM. So, if an attacker knows of a page on your website that consumes a lot of memory, then the attacker can simply release some bots on that page and quickly bring your server to its knees by exhausting all of its (the server’s) memory.

But what is an ideal PHP memory limit?

Ideally, a page on a Joomla website should not consume more than 32 MB of memory (and that’s very generous). If you find that on some pages (with the exception of the core update page) your Joomla website needs more memory than that, then you might be a victim of a bad Joomla programmer. In that case, you will need some Joomla experts, such as (ahem) us, to address this issue for you. Just contact us and and we’ll fix the problem for you in no time and for a very reasonable price!

The Dangers of Relying on Joomla’s Banners Extension for Advertising

We just got a call from a new client, telling us that she was perplexed about the fact that the number of banner impressions on her Joomla website is unrealistic. She said that her Joomla website is getting about 10K visitors/month according to Google Analytics, but the number of impressions counted on her Joomla website are almost double that number.

Our immediate answer was: “yes – this is normal”, to which she replied: “How?”, and we replied: “Google Analytics only counts the net traffic to the website, so, it excludes all bots and spam traffic. Joomla doesn’t have the mechanism in place to discount that traffic, and so the traffic figures (from Joomla’s perspective) are highly inflated – hence the huge discrepancy.”

“Bummer!”, she said, “what are we going to do? We need to sell advertising to potential clients and we don’t have the correct number of impressions. Is there anything we can do now?”, she asked. We told her that for now she can communicate the numbers to the advertisers along with a disclaimer that these numbers may be highly inflated, but for the future, that she must use Google DoubleClick to manage her ads. The nice thing about Google DoubleClick is that it displays the exact number of impressions, and the exact number of clicks, and it automatically excludes all non-desirable traffic from the actual traffic numbers. The other nice thing about it is that it’s free (unless the website serves more than 90 million impressions/month – in which case publishers must pay money for Google).

So, does the above mean that the Joomla Banners extension is useless?

Yes – it does – we can’t think of a single reason which makes the Joomla Banners extension a practical ad management tool in today’s world wide web (in fact, two years ago, we have stated that this extension must be removed from the Joomla core). Maybe it was useful back in the early 2000s (back in the Mambo era), but, in 2016, there is not one reason for such an outdated extension to exist. Unless Joomla plans to integrate a webspam engine in its core, then the Joomla banners extension should no longer exist in the core.

If you are currently using the Joomla Banners extension and you just discovered that its statistics are way off and you want to implement Google DoubleClick, then we’re here to help. All you need to do is to contact us and we’ll take care of the whole thing swiftly, cleanly, and affordably!

SQL Injection in Joomla – Is It Still a Concern?

The last time we had a case of SQL injection on a Joomla website was a long time ago – and the affected Joomla website was a Joomla 1.5.10 website (which is highly exploitable – even by Joomla 1.5.26 standards). Since then (we are now in October of 2016), we have not seen a single case of SQL injection on a Joomla website.

Does that mean that Joomla is more secure now?

Well, Joomla is much more secure now than anytime in the past, but that is not the only cause leading to the significant drop of SQL injection attacks on Joomla websites. In fact, there are other, more important, causes for this drop:

  • Security tools on servers becoming mainstream: A few years ago, ModSecurity was installed on a very small percentage of servers. Nowadays, almost all servers come with ModSecurity installed and enabled by default (hence the increase of quirks on Joomla websites caused by ModSecurity). ModSecurity is excellent in blocking patterns that may lead to a SQL Injection attack.
  • A more rigid process for accepting Joomla extensions: Joomla extensions were notorious for being insecure. But, as of a couple of years ago (when the JED was revamped), the once lax process of accepting and testing Joomla extensions evolved into a more serious and rigid process, quickly rejecting extensions that do not meet the Joomla security standards. Additionally, the process of immediately suspending vulnerable extensions (and publishing them on the the VEL [the Joomla Vulnerable Extensions List]) has forced developers to ensure that all user input is filtered properly in their extensions.

  • A lack of attackers’ interest in SQL injection: Attackers follow the trends, and SQL injection is no longer the trend. In fact, the current trend is to initially upload a few files and then overwrite some core files, ultimately controlling the website and use it to attack other websites and/or use it to download malware to innocent visitors’ computers. Attackers are simply no longer interested in altering the database in any way (for unknown reasons), even though they could (if the attacker is able to upload one file – just one file – to a website, then he can potentially read the main configuration file and then gain easy access to the database).

We think that the last point is the most important one of the lot – it is the mysterious absence of interest in modifying the database (by the attackers) that has contributed the most in the huge drop of SQL injection attacks. One would think that the attackers have a syndicate that decides which types of attacks are allowed and are not allowed – and it seems that some time ago, the syndicate has decided that SQL injection attacks are no longer permitted.

Whatever the real reason behind this is, we hope that SQL injection attacks remain very low on Joomla websites, because database hacks are the worst type of hacks on any website.

So to answer the question of this post – is SQL Injection in Joomla still a concern? – then the answer is yes – it is – simply because vulnerable websites can still be easily SQL injected – but it’s just that the attackers have elected to stop doing it, and it may be only a matter of time before they revert back to their old habit.

Now if you, our dear reader, are one of those unfortunate Joomla administrators whose website experienced the worst type of hacks, then fear not, all you need to do is to contact us. We’ll cleanup your website and we’ll secure it for you quickly, efficiently, and affordably!

HubSpot Adding Weird Tracking Code to Links on a Joomla Website

Some time ago, while performing daily maintenance work for one of our large clients, we noticed that some of the external links that they had had some weird hash in them. What was interesting is that all these external links that had this problem were domains owned by the client. A little digging into the HTML code unveiled that the problem was caused by HubSpot.

We communicated this issue to HubSpot, and they confirmed to us that this is because the client elected to track the outgoing traffic from their main website to sister websites, and that was the only way for HubSpot to do the cross-domain tracking. At that time (that was months ago), we told HubSpot that we were concerned that these weird hashes may cause content duplication issues, as they may get indexed by Google. They replied that this shouldn’t have a problem whatsoever, since all the pages on the website had canonical URLs, which means that these weird hashes will not result in many variations of the same pages, which in turn means that Google will not index those variations.

Fast forward to a couple of weeks ago (yes – we prepared this post a couple of weeks ago – but we didn’t have the time to publish it until today), where it was a completely different story. The moment we logged in to the Google Search Console of that particular website, we discovered that thousands upon thousands of pages such as http://www.[ourclientjoomlawebsite].com/page-1.html?__hstc=[long-hash]&__hssc=[medium-hash]&__hsfp=[small-hash] (where long hash, medium hash, and small hash were long, medium, and short random strings) were 404 pages. Huh?

After analyzing the issue while in super panic mode, we discovered the sequence of issues that were causing this mess:

  • The Google bot visits a page on the website.
  • The visited page contains a link to a non-existent page on a sister website.

  • The link has the HubSpot tracking code appended to it.

  • Google tries to visit the link and thus results in a 404 error for that link.

  • Since the link is a 404 link, then the whole canonical concept does not apply to it, which means that Google ends up visiting many variations of that same 404 link, and thus resulting in many 404 errors. This is a huge issue because it highly inflates the 404 count of the website (which has negative SEO impact.

Of course, the root of the problem was the 404 link, but the problem was amplified by HubSpot‘s tracking code. We brought up this issue with the client, and we presented them with a couple of options:

  1. We fix the 404 links – which means that we will risk having the same series of events in the case of new 404 links.
  2. We disable tracking on auxiliary (sister) websites.

The client went with the second option, mainly because they were completely disturbed by all these weird hashes they were seeing on their website. As usual, we happily obliged! Here’s how:

  • We logged in to the HubSpot account of the client.
  • We clicked on Reports -> Reports Settings.

  • Under Domains, we unchecked the Enable checkbox under Automatic cross-domain linking.

  • We saved the form and then we checked the website, and hooray, all these annoying hashes were gone!

We think that HubSpot‘s strategy for tracking outgoing traffic is somehow flawed: the whole concept of adding weird hashes does not fly well on serious websites. Additionally, as we have demonstrated above, this strategy can cause SEO issues.

If you see weird hashes in some of your URLs after adding HubSpot to your website, then you should uncheck the Automatic cross-domain linking feature in HubSpot. If you need help in doing that, then just contact us and we will gladly do it for you. Our fees are affordable, our Joomla expertise is second to none, and we will strive to be your friends (but not in a cable guy kind of way).

Yet Another Login Loop in Joomla’s Backend

Login loops are probably the most complex problems to fix on a Joomla website. Here’s the scenario: you go to the login page of the backend of your Joomla website, you enter your username and password, you click on the blue Log In button, but, to your surprise, you are redirected back to the login page with no error.

The thing is, we have encountered this problem many times before, and still, each time a client tells us that he has this issue on their website, we swallow about 8-10 aspirins, and we brace ourselves for a very, very long night… Unfortunately, yesterday (a Sunday), we were unlucky enough to work on this issue…

Yesterday started as very beautiful day, very sunny and with a very blue sky (which is not very common in Montreal), but it got cloudy pretty fast when a new client contacted us with the most terrible news: he wasn’t able to login to the backend of his Joomla website, and every time he tries to login he is redirected back to the homepage.

The reason why the news were that terrible is because we have yet to see two of these login loop problems caused by the same root issue. Each of these login problems is unique, as well as its solution. Still, were started the work with a healthy amount of optimism. But, as we later learned, optimism is a fickle friend, who jumps a sinking ship at the nearest opportunity!

We started the work by trying to find whether the cause of the problem is one that we have encountered before:

  • Is the tmp physical path (the $tmp_path variable in the configuration.php file) correct and writable by the web server?
  • Is the log physical path (the $log_path variable in the configuration.php file) correct and writable by the web server?

  • Is the cookie domain (the $cookie_domain variable in the configuration.php file) empty?

  • Is the plg_user_joomla plugin enabled? Is the plg_authentication_joomla plugin enabled? (Note that we used phpMyAdmin to check for the value of these two plugins in the #__extensions table).

  • Is the session handler (the $session_handler variable in the configuration.php file) set to database?

  • Are all types of cache disabled?

As you may have probably expected, all the above were OK – if they were not OK, then what is the point of this post? So we decided to take a more aggressive approach to solve the problem…

We started our aggressive approach by overwriting the Joomla website with a fresh copy of the same version of Joomla using our super quick famous technique for cleaning up Joomla websites, but that didn’t work!

So the next step in our aggressive approach was to check the controller.php file (located under the administrator/components/com_login folder) which is the entry file responsible for logging in the user (there are other, deeper files that effectively handle the login, but this is the entry file). A thorough debugging of the file revealed that the problem lied in this line:

JSession::checkToken('request') or jexit(JText::_('JINVALID_TOKEN'));

If you’re a close follower of our blog, then you may remember that we have published several articles on Joomla’s invalid token error. In fact, in one of our blogs, we concisely explained how to get rid of the invalid token error once and for good. But, we didn’t want to do apply this fix for that client because 1) he was using the latest version of Joomla, and 2) there shouldn’t be a reason for him to see that error on his website in the first place, and 3) this is a last resort solution.

So, obviously, the issue is caused by a problem with the session storage, but we have already checked everything – literally everything having to do with the session, so why do we have a problem with session storage? At this point, we were only able to think of one culprit: a Joomla update that left some reminiscent files from a previous version… So we compared the files of the Joomla website with those of a fresh Joomla website (having the same version), and we noticed that there were some extra files/folders in our client’s websites (reminiscent files from a previous Joomla version). We deleted the extra files and folders and we tested this problem again, but still, it didn’t work.

In order to break the series of unfortunate events (yes, we are alluding to the movie here), we called our moms so that they can give us much needed morale, and they did. We then relaxed for a bit and then started attacking the problem again! We thought, hey, there is this one file that we haven’t checked yet and that we credit with a not-so-small percentage of all the evil in the world (let alone Joomla): it is the .htaccess file. So, we opened the .htaccess file, and, in its very beginning, we saw the following 2 lines:

Header set Set-Cookie "VC-NoCache=1; max-age=900; path=/"
Header set VC-NoCache "1"

Huh? What is that, we thought? Is that something that has to do with caching? Let’s just comment out these 2 lines and see if that takes care of the problem, and it did! We were able to login without being affected by the vicious and ruthless login loop. We were happy but we were tired – it was a frustrating and a scary experience!

If you, our dear reader, are on the brink of deleting your whole Joomla website (and business) because of this error, then please check our previous posts about this issue (which are linked to in the beginning of this article), and also make sure that you don’t have something in your .htaccess file causing this problem. If you are not able to solve this problem, then just contact us and we’ll take care of it. But, before doing that, please make sure you check our fees and please don’t forget to send us a blindfold and a cigarette – we may need them!

“500 – Unable to load renderer class” Error on Joomla

A Joomla designer called us this morning and told us that he was seeing the following error when trying to load the Joomla website of his client:

500 – Unable to load renderer class

He told us that this whole thing happened after switching to a new template, and that reverting back to the old template did fix the problem for him (but he really needed the new template to work). This little piece of information was super useful for us, as it narrowed down our search area from all of the Joomla website, to a specific template.

So, we opened the main index.php of the problematic template, and it didn’t take us long to figure out where the problem lied. It was this line:

<jdoc:include type="logo" name="modules" style="none" />

The thing is, logo is not a jdoc type – it is a jdoc name. So, we changed the above line to this one…

<jdoc:include type="modules" name="logo" style="none" />

… and that fixed the problem!

But what are the allowed jdoc types?

In case you’re wondering, the allowed jdoc types are the following:

  • component
  • head
  • message
  • module
  • modules

If one of your jdoc types is not not in the above list, then you are likely to see the “500 – Unable to load renderer class” error on your Joomla website.

We hope that you found this very shot post helpful (we’re not fond of short posts at itoctopus) and we hope that it solved your problem. If it didn’t, then please contact us. We are always glad to help, our work is super professional, and we don’t charge much.

You Can’t Receive Emails to Your Own Domain from Your Joomla Website? Read This!

Last Monday, we received an email from a company (which is a client of ours) stating that any email sent from their Joomla website to an email to their domain isn’t being received. For example, assuming that their Joomla website is ourclientjoomlawebsite.com, and the Joomla website tries to send an email to admin@ourclientjoomlawebsite.com (the email, in the case of our client, was a notification email from an RSForm form), then that email doesn’t get delivered.

When they first told us about the problem, we insisted that the problem was caused by missing SPF settings, and so we told them to add the IP of their server to the SPF record in their DNS (they were using Office 365 for their mail), but they didn’t do that. The next day (Tuesday), the IT Manager working for the client called us and told us that the email wasn’t even caught in a spam filter by the mail server – so, most likely – the email wasn’t even sent by the server which was hosting the Joomla website, so it’s definitely not an SPF issue. He then mentioned that they receive notifications from their Joomla website to Gmail and Yahoo accounts, but they don’t receive them when they are sent to company emails. We thought, that was strange, but we insisted that they add the IP of the server to their SPF records, just to see if that solves the problem. Finally, they did – but, to our regret (and shame, and humiliation, and embarrassment, and disgrace, and…), that didn’t fix the problem.

The IT Manager then emailed us the negative results, urging us to see if the issue is caused by RSForm, the Joomla website, or the server. So we did (today), and the first thing that we examined was the exim logs the following way:

  • We ssh’d to the server.
  • We changed to the /var/log/ directory by issuing the following command:

    cd /var/log

  • We searched the log for any entries pertinent to the affected email address:

    grep -R "admin@[ourclientjoomlawebsite].com" *

  • We saw the following results:

    exim_mainlog:2016-09-23 15:11:20 1bn9Oy-000DDQ-73 <= no-reply@[ourclientjoomlawebsite].com U=**** P=**** S=1662 id=cfc73da22404d08163d64fbbab9316b5@www.[ourclientjoomlawebsite].com T="New submission from 'Contact Us'!" for admin@[ourclientjoomlawebsite].com exim_mainlog:2016-09-23 15:11:20 1bn9Oy-000DDQ-73 ** admin@[ourclientjoomlawebsite].com R=virtual_aliases: No Such User Here

Hmmm! Looking at the above was enough for us to explain what was going on. The server hosting the Joomla website had its own mail server, and that server thought that it was responsible for all email accounts belonging to ourclientjoomlawebsite.com (which was wrong, since Office 365 was responsible of these email accounts), and so the exim mail application was checking the internal (server) database for the existence of these accounts, and then silently failing when it’s not able to find them.

So, how did we fix the problem?

Fixing the problem was quite easy:

  • We logged in to the cPanel account of the domain.
  • We clicked on MX Entry under Email.

  • We selected our client’s domain and we chose Remote Mail Exchanger in the Email Routing section.

  • We clicked on the Change button.

  • That’s it!

After doing the above, we tried sending a notification email from the Joomla website (we did that by filling out an RSForm form), and, guess what? It worked! In fact, we knew that it worked because the client sent us an email a few seconds after we submitted the form telling us that the problem was solved! We were happy and the client was happy, which is always ideal in our line of job!

How come the fix worked immediately?

Technically, the fix didn’t consist of any DNS changes which usually take time to resolve. The fix consisted of a small setting change on the server, telling it that it’s not responsible for routing emails (and that the routing of emails is handled by an external server). These kind of setting changes take effect immediately, and they don’t even require restarting the mail server.

What about our ego?

We admit it – we started this task with an inflated ego. We were confident that the problem was caused by SPF settings and we didn’t listen carefully to what the client was saying. That was a mistake but the consequences of that mistake were not. In fact, we were humbled when we found out that we were wrong (after insisting that we were right), and we were gently reminded that an inflated ego attitude is a bad thing in our business, and that we should always, always, listen very carefully to the client.

We hope that you found this post informative and that it helped you solve your problem. If it didn’t, then please make sure that your SPF settings are correct. If they are, or if you feel that this whole thing is a bit over your head, then fear not, we’re here to help! Just contact us and we’ll take care of this issue for you in no time and for a very affordable fee!

The Illogical Ordering of Articles on Joomla 3.6.+

Note: This post is a rant! Proceed at your own risk!

When Joomla 3.6 was released, most of those who updated their websites started complaining: I added a new article, but I can’t find it anywhere on my website. What is the problem?

The problem is that someone (you know yourself if you’re reading this!), who suddenly decided to become a Joomla core developer (that someone has never developed anything on Joomla before), enhanced the adding of new Joomla articles by giving newly added articles the last possible ordering. In other words, let’s say that you have 100 articles, all in the same category. When you add a new article to your website, the article ordering of the new article is 101.

Now, that someone (we will not refer to him as a developer and we will explain later why), thought that he made a breakthrough in the science of Joomla: by using very little code he made the process of adding new articles much faster on large Joomla websites. How is that? Well, in Joomla versions 3.5.1 and prior, when a new article was added, it was given the ordering “0”, and all the other articles’ orderings were shifted by 1. The shifting part is an expensive process in MySQL, especially if the website has many articles. By avoiding the shifting process, the adding of new articles became immensely faster.

But, what is the problem?

Well, the problem is that this enhancement breaks the default ordering functionality in Joomla. So, if an administrator adds an article to his Joomla website and is using the Article Ordering sorting feature for ordering his Joomla articles, he won’t be able to see that article neither in the frontend nor in the backend unless he browses to the last page.

This has caused a huge issue in the Joomla community, with many people complaining that they can’t find their new articles in Joomla 3.6.+ (or that they can’t add new articles in Joomla 3.6.+), while in reality, their articles were there, but they’re just ordered last. Unfortunately, that someone who broke the ordering functionality refused to admit that it was a mistake – claiming that (and we’re quoting his exact words) “it is wrong to use the field ordering to get content sorted by a date”. Which brings us to the next question:

Why do we refuse to call that “someone” a programmer?

Well, he is not a programmer for the following reasons:

  • He failed to understand that the code to shift all the articles when a new article was inserted was there for a reason. He happily erased many lines of codes and just added a few lines without even analyzing the original code. Any programmer should thoroughly examine the code he’s replacing before doing anything.
  • He didn’t test his code on a real large website. How did we know that? Because if he tested his code on a real large website he must have noticed the issue. But he didn’t (notice this issue), and so we are assuming good faith here that he didn’t test his super enhancement. The irony in this whole thing is that his code was meant for large websites. Small websites don’t suffer from any performance issues because of the shifting process.

  • Even after seeing the reports that his new enhancement broke many Joomla websites out there, he stuck to his code change claiming that all these Joomla administrators were using the wrong ordering, which is, in fact, a wrong statement for 2 reasons: 1) there is no such thing as wrong ordering in Joomla (those administrators were just using an existing sorting option in the backend), and 2) even if that’s the case, he shouldn’t develop something that breaks websites. In fact, he only yielded when some heavyweight contributors in the Joomla core were involved in this issue, while still claiming that reverting back was the wrong thing to do.

  • He wasn’t quick in cleaning up his own mess. A programmer, a real programmer will immediately revert his changes if they negatively affect the product. The Joomla community has to wait until Joomla 3.6.3 in order to see this issue fixed (though, in all fairness, we cannot blame this delay totally on him).

The reason why we have strong feelings about this is that the actions of this “someone” caused many people using Joomla to lose some of their trust in the product. His actions were immature at best, and destructive at worst. There are people who are born to be developers, and there are people who are born to be core developers, and this person is in neither group.

In our opinion, he shouldn’t have been allowed to touch the Joomla core, and this brings us to a few existential Joomla questions: Who authorizes work on the core? Who authorizes who should work on the core? And who approves core changes? If that kind of extremely weak code was let through, then this means that the answers to these 3 questions must be all revised.

We are always trying to make Joomla better, hence the rant. If you are reading this and you are a core Joomla developer, then please understand our true motives – we are not against you, we are against bad code especially in the core. We respect the work of all of those who are involved in Joomla, and who are considered, by programming standards, to be programmers.

Now, we turn to you, our dear readers. If by any chance you are reading this post because you updated to Joomla 3.6.+ and you can’t see your new articles, then fear not, we are here to help! Just contact us and we’ll fix the problem for you in no time and for a very affordable fee. We will also make sure that our fix is update proof!

The Malicious “security.php” File on Joomla Websites

We are currently getting swamped with hacked Joomla websites with a malicious security.php file in their root directory (e.g. at the same level of the index.php file). The name of that file is extremely misleading because it implies security, while, in reality, it is the complete opposite. In fact, Joomla websites should not contain a file carrying this name anywhere (not in the root directory, not even in any subdirectory).

Ever since August 20th, 2016 (it was when we saw the first occurrence of the security.php file), we are seeing more and more of these malicious files on Joomla websites.

So, how is this file uploaded onto the Joomla website?

In most cases, the file was uploaded using an exploit on the Joomla website, and that exploit is typically a vulnerable extension. We know that because the absolute majority of Joomla websites infected with that file are running the latest Joomla version (3.6.2 at the time of writing this article), which is known to be secure. In some cases, the file gets uploaded because of a vulnerability on the LAMP server, but again, the absolute majority of the time the culprit is an outdated extension.

So, what does this file do?

The security.php file is a main backdoor file, that is used to control the Joomla website (even when all the other exploits [vulnerable extensions, outdated Joomla website, vulnerable server] are closed) by allowing the attacker to execute remote commands comfortably. The fact that it has this trustworthy name lets it go unnoticed by many Joomla administrators, who are misled into thinking that this file is actually about security, and deleting it might compromise the security of their websites. In fact, we had several people emailing us about this file as they have never seen it before (almost all of them were convinced that it was part of Joomla 3.6.2 security).

So, what can someone do to protect his Joomla website from that file?

The best way to protect a Joomla website from the malicious security.php is to prevent the upload in the first place by closing all the possible exploits, which consists of: updating the Joomla website to the latest secure version, removing unnecessary and vulnerable extensions, updating all the extensions, and ensuring that the environment is secure (e.g. Linux, Apache, MySQL, and PHP do not have any vulnerabilities).

There is also a fallback protection method, which consists of only allowing the index.php to be access from the outside world as described here.

If you ever see a security.php under the root directory of your Joomla website (or anywhere on your Joomla website, for that matter), then you should resign to the fact that your website is hacked and you should proceed accordingly (you should clean it up and secure it). If you need help with unhacking and securing your Joomla website, then look no further: we are the Joomla security experts! Just contact us and we’ll ensure that your website has stellar security in no time and for very little money!

“Ajax Loading Error: Category not found” Error when Updating a Joomla Website

A new client, with a Joomla 3.6.0 website, called us yesterday evening and told us that every time he tries to update his Joomla website, he sees the following the following error:

Ajax Loading Error: Category not found

So, we went to his website, and tried to update Joomla, but, as expected, we saw the above error, in a JavaScript popup. Hmmm! Debugging these JavaScript problems is typically not easy, we thought… But we promised the client that we will find a solution for him!

So, the first thing that was on our mind was cache: we disabled all types of cache on the website (global cache and system cache), and then we cleared the cache (Joomla cache and browser cache), and then we tried to update the website again, but we still saw the above error.

We searched for the error in the Joomla code base, in hope of finding the exact location of the problem, but we couldn’t find anything. To be honest, it’s not that we didn’t find anything, we did find something, but what we found wasn’t really helpful in getting to the root of the problem. So we decided to stop beating around the bushes, and instead, we took an approach that was completely against our programming habits: debugging Ajax code (nobody likes to debug Ajax code).

So we opened the JavaScript file containing the Ajax code responsible for throwing that error, which is the update.js file (which is located under the media/com_joomlaupdate/js folder) and we checked its code. After a thorough examination, we knew that for us to know what is really going on, we needed to find which PHP file is processing the Ajax request. And so we changed the following line in the update.js file:

var message = 'AJAX Loading Error: ' + req.statusText;

to this line:

var message = 'AJAX Loading Error: ' + req.statusText + ' update URL is: ' +  joomlaupdate_ajax_url;

We then cleared our browser cache (again), and tried to update the website, and this time, the JavaScript alert window displayed the URL that the Ajax request was being sent to. It was this one: http://www.[ourclientjoomlawebsite].com/administrator/components/com_joomlaupdate/restore.php

So, we visited the URL directly, and it had the following error:

404 – Category not found

That was odd, because the file restore.php existed. So we added the following code to the very beginning of the restore.php file (just after the opening tags):

die('In restore.php file');

We then visited the above URL again, but we were surprised to see the same error again. This meant 2 things: either there is an .htaccess rule whitelisting specific PHP files, or the file permissions/ownership were wrong. So we first checked the .htaccess file and we noticed that it had the following code in its very beginning (probably copied from here):

<Files *.php>
	Deny from all
</Files>
<Files index.php>
	Allow from all
</Files>

The above code secures the Joomla website by allowing direct access to the just one file, which is the index.php file. On the flip side, it doesn’t allow direct access to the restore.php file, which is a legitimate file used for the update.

So, we updated the above code to the following to allow access to the restore.php file…

<Files *.php>
	Deny from all
</Files>
<Files index.php>
	Allow from all
</Files>
<Files restore.php>
	Allow from all
</Files>

… and then we visited the website, but still, we saw the same error. So we quickly checked the permissions/ownership on that file and we noticed that both the permissions and the ownership were incorrect: the permissions were set to 600 (note that we didn’t notice this when editing the file because we edited the file while ssh’ing as root), and the ownership was set to root. We suspect that that file was infected previously and so whoever did the cleanup decided to just make the file completely inaccessible (with the exception for the root user) after the cleanup process. So we changed the file permissions to 444 and we changed the ownership and the group ownership to the cPanel user of the website. We then visited the file, and this time, we saw the message that we added in the restore.php file. We then removed the die statement that we have added, and we tried updating the website, and, as expected, it worked smoothly! Hurray! Lush homes all around! (In case you’re wondering, that Lush homes thing is from the game Tales of Monkey Island).

So, if you, our dear reader, are seeing the Ajax Loading Error: Category not found popup when trying to update your website, then fear not: check if your .htaccess file is allowing access to only specific files and also check the ownership/permissions on the restore.php file. If everything checks and you are still having the problem, then it’s time to contact the Joomla experts! We are there for you 24/7/365, we know our Joomla, and our fees are super affordable!

Blank Page When Clicking on “Read Messages” in Joomla’s Backend

A client contacted us on Friday of last week and told us that he had the following weird problem: every time he clicked on Read Messages (which is located in the You have post-installation messages box) in the backend of his Joomla website, he saw a blank page.

A blank page in Joomla, as you may already know, is a definitive sign of a PHP fatal error somewhere… So, in order to see the error, we set the Error Reporting value to Maximum in the configuration settings, and then we clicked again on the Read Messages link, and we saw the following error:

Fatal error: Class ‘FOFRenderDirs24’ not found in libraries/fof/view/view.php on line 966

Hmmm! What is that FOFRenderDirs24 class? Something smelled very fishy as we have never heard of that class before…

In any case, we checked the line 966 of the view.php file (which is located under the libraries/fof/view folder), and its job was to instantiate an object from a PHP class which name is based on appending the word FOFRender with the name of each and every file name in the libraries/fof/render folder. For example, if the libraries/fof/render folder had two files, one is called first.php, and the other is called second.php, then line 966 would instantiate two objects, one from class FOFRenderOne and the second from class FOFRenderSecond, which means that the first file must define a class called FOFRenderOne, and the second file must define a class called FOFRenderSecond (yes, we know, it’s starting to get confusing a bit – just don’t ask to repeat this paragraph).

Now, the fact that line 966 was failing when instantiating an object from class FOFRenderDirs24 meant only one thing: that there is a file called dirs24.php in the libraries/fof/render and that that file doesn’t define a class called FOFRenderDirs24. So, we checked the libraries/fof/render, and we noticed that we had six files in there, instead of 5 on a standard Joomla 3.6.2 website, and one of them was a zero byte file called dirs24.php. As you might have probably guessed, the dirs24.php was the intruder file, and so deleting that file fixed the problem.

But where did that file come from?

As we have mentioned earlier, the website was previously hacked and freshly cleaned. The cleanup work was done by an automated tool, which emptied files with purely malicious content. The thing is, with Joomla, emptying files (instead of deleting them) is a bad strategy, as the Joomla engine, in certain directories, automatically tries to define classes (and instantiate objects based on those classes) based on the filenames in those directories.

So, if your Joomla website is showing a blank page or a fatal error when you click on that Read Messages button, then you should check the libraries/fof/render for any intruder. If there isn’t any, then the problem might lie elsewhere, and you may need some Joomla experts for help. So go ahead, contact us, we will solve the problem for you, and we won’t charge you an arm and a leg (or even a toe) for it!

Checkboxes Not Showing in the Backend of a Joomla 1.5 Website

Very early in the morning today, a new client sent us the following email:

“Hi,

We have a Joomla 1.5 website that has been working well for a long time. Yesterday, our host decided to upgrade PHP, which resulted in the following problem on our website: checkboxes are no longer visible anywhere in the backend of our Joomla website. We used these checkboxes to select articles (in the Article Manager) for deletion – but now we can’t delete articles this way anymore.

Is there a fix that does not involve reverting to an earlier version of PHP?”

When we read the email, we quickly remembered our post on the hidden menu manager issue in Joomla 1.5, which is also caused by a PHP upgrade.

So, naturally, the first thing that we did was that we enabled error reporting on the Joomla 1.5 website, and then we went to the Content -> Article Manager page where we were greeted with the following 2 errors (repeated about 20 times):

Warning: Parameter 1 to JHTMLGrid::access() expected to be a reference, value given in libraries/joomla/html/html.php on line 87
Warning: Parameter 1 to JHTMLGrid::checkedOut() expected to be a reference, value given in libraries/joomla/html/html.php on line 87

Aha! So the root cause of the problem was very similar to the root cause of the “hidden menu items issue”: it was an expected reference but value given issue. So fixing the problem was actually very easy; all we did was the following:

  • We opened the file grid.php which is located under the libraries/joomla/html/html folder.
  • We changed the following line:

    function checkedOut( &$row, $i, $identifier = 'id' )

    to:

    function checkedOut($row, $i, $identifier = 'id' )

  • We also changed this line:

    function access( &$row, $i, $archived = NULL )

    to this one:

    function access($row, $i, $archived = NULL )

    Notice the missing ampersand (&) from the second, correct line.

  • We saved the file and then refreshed the Article Manager page, and this time, all the checkboxes displayed and worked properly, and all it took was the removal of an ampersand (well, two ampersands, but you get the point)!

But isn’t that a core change?

Yes – the solution presented in this post is a core change, but it’s Joomla 1.5, which will never ever get an official update, so who cares? In fact, we think that modifying Joomla 1.5’s core is a must for those seeking security and stability but cannot afford to migrate to the newest version of Joomla.

If you have the same problem on your Joomla 1.5 website, then try our solution above, it should work for you! If it doesn’t, then please contact us. We will help you fix the problem quickly, cleanly, and affordably!

Alternative Layout Not Working for Your Custom Joomla Module? Read This!

While working on making a very large website responsive over the past few days, we were faced by a small blip. A small, but interesting (and annoying) blip… We wanted to show a different layout for a custom made module (we’re not talking here about Joomla’s Custom HTML module [which is now called just the Custom module]; we’re talking about a module that was specifically created for that website), and so we just created the appropriate folder under the html folder of the template being used, and then we copied the layout file of that module (default.php) to the folder that we have just created. After doing so, we did the modifications that we needed to do on the copied file (we called it mobile.php), and then we went to the Joomla backend and tried to select the new layout that we have created from the module’s settings. But, the field for selecting a layout was just not there.

No problem, we thought, as we knew the exact cause of this issue: the Alternative Layout field did not exist in the XML manifest file of the custom module. So we opened the XML manifest file of a standard Joomla module, and we copied this line…

<field
	name="layout"
	type="modulelayout"
	label="JFIELD_ALT_LAYOUT_LABEL"
	description="JFIELD_ALT_MODULE_LAYOUT_DESC" />

…to the manifest file of the custom module (just before the closing fields tag). After doing this, we went back to the module, and confidently chose mobile, and then we went to the website, and checked to see whether our changes to the layout of the module have taken effects, but they didn’t. Easily fixed, we thought… We went back to the backend and cleared the Joomla cache, but still, the website still showed the default layout for that module.

We went back to the module, and we tripled checked that it was that module being loaded, and that the layout change was saved, and everything checked. Naturally, clearing the cache a few dozens time more didn’t fix the problem – it just wasn’t a cache problem!

We finally decided to check the module’s main PHP file, which is module_name.php under the module’s folder, and its last line (responsible for calling the file that does the actual display) consisted of the following:

require(JModuleHelper::getLayoutPath('mod_modulename'));

But, in order for the module, any module, to benefit from the Joomla alternative layout functionality, it must have this line at the end (instead of the one above):

require JModuleHelper::getLayoutPath('mod_modulename', $params->get('layout', 'default'));

So we switched the last line of the custom module with the line above, and this time, the layout override worked! It worked! It was a great relief mixed with some irritation, because we knew that the root cause of this problem was a bad Joomla developer. In any case, we were exalted because the problem was solved.

So, if your alternate layout is not displaying on your Joomla website, make sure that your module supports alternative layouts both in its XML manifest file and in its main PHP file. If it doesn’t, then you can resolve this by following the tips in this post. If you need help with the implementation, then please contact us. We will help you address this in no time and for a very affordable fee!

A Very Long Password in Joomla Does Not Mean Better Security

A new client of ours called us very early in the morning and told us that his website was hacked and he wanted our help to clean it. He told us over the phone that he had no idea how he got hacked despite the fact that he had a very long and complex password for his super user login. He then sent us the password… It was literally 60 characters long with every almost every single allowed character in it. It was the first time we saw something like this. The longest we’ve seen so far was around 32 characters, but 60 characters? So we told the client (he was still talking to us over the phone) that having a very long and very complex password will not make his website unhackable – in fact, it has nothing to do with website protection. The best that such a password can do is protect the website against dictionary attacks, which is a malicious method to login to a website using dictionary words (note: dictionary attacks are never used by serious attackers because they are inefficient).

As expected, he asked us: “Then how did they get in into my Joomla website?” So we pointed him to our famous “10 Reasons Why Your Joomla Website Got Hacked” article, while at the same time explaining to him that they got in most likely by exploiting a vulnerable extension on his website (his website was using the most recent Joomla version, but many of its extensions were outdated).

So, is having a very long Joomla super user password a bad thing?

It’s not a bad thing, but it’s not a good thing either. We have been supporting Joomla for a decade now, and this is how much we’ve seen a Joomla website got hacked because of a simple password: 1 – yes, one time, just one, and it turned out that the person who hacked that particular website was a previous disgruntled company employee.

An attack on a Joomla website is done nearly all the times by exploiting a server vulnerability, or an application vulnerability, or a combination of both. It’s almost never done by using tools that just “guess” what the password is.

If your Joomla website got hacked and you cleaned it up, then using a very long password for your super user will not protect it and will not make it harder for malicious users to compromise your website. If you need real protection on your Joomla website, then follow our Joomla security tips. If you need super advanced (enterprise level) protection for your Joomla website, then please contact us. Our fees are right, our security expertise in Joomla is undisputed, and we always welcome new clients (and new friends)!

White Screen when Trying to Edit a Joomla Article: How to Fix

It’s Friday – time for everyone to relax a bit and plan for the weekend! Everyone, of course, except for us! We work 24/7/365! In fact, last Sunday, a client emailed us and told us that they have a problem when editing articles in Joomla’s backend: whenever they click on an article to edit it, they see a blank page.

Now a blank page is a sign of a fatal error somewhere, so naturally, we had to enable error reporting on the Joomla website. So we logged in to backend (of the website), and then we went to System -> Global Configuration, and then we clicked on the Server tab, and we changed Error Reporting to Maximum. We then tried to edit an article to see what the error was, and we saw this:

Fatal error: Call to undefined method WFBrowserHelper::getMediaFieldLink() in /plugins/system/jce/jce.php on line 37

Hmmm! It turned out to be an error with JCE, which is, by far, the best Joomla editor out there, but which is also extremely complex. The client was running the latest version of Joomla (3.6.2), and so we thought that it wouldn’t be a bad idea to uninstall JCE and then re-install it. And so we did, and then we tried to edit an article again (after clearing the Joomla cache and the browser cache, just in case), and this time it worked! It really worked! The client was happy that we solved the problem quickly and we were happy that it turned out to be a very easy fix!

But what caused this problem?

We don’t know – we think it might have been a compatibility issue between the JCE version that the client had and the latest version of Joomla, but we’re not really sure. The client didn’t mention when the problem started happening and we didn’t ask (now we’re kinda regretting this).

So, if you’re seeing a white screen when you’re trying to edit a Joomla article (or if you’re seeing the error above), then try uninstalling JCE and then reinstalling its latest version onto the website and see if that fixes the problem. If you don’t have JCE, then try updating the editor that you’re using. If you’re using a core Joomla editor (such as TinyMCE or CodeMirror), then it might be that the problem is caused by a 3rd party plugin that you have just installed – try disabling your plugins ones by one (starting with the one that has the highest ID) until you find the culprit. If you can’t find the culprit, then you can always contact us. We’ll fix the problem for you in as little time as possible and for a very reasonable price!

“You are not permitted to use that link to directly access that page” Error when Trying to Edit a Joomla Article

An important client of ours emailed us yesterday and told us that they were having the following problem: whenever they edit an article, save it (or close it), and then try to re-open it, they are redirected back to the Articles Manager page (instead of being redirected to the edit page of the article). They told us that the issue was only happening on Google Chrome.

So we tested the website, and we were able to replicate the issue (only on Chrome), with one difference: the first time we tried to re-open any article, the Joomla CMS was throwing the following error:

You are not permitted to use that link to directly access that page (#[article-id])

We know, from experience, that that error is caused by session issues, so we checked everything that had to do with the session in the code…

First, we checked the file controller.php (which is located under the administrator/components/com_content/ folder) where the following code was responsible for throwing the “You are not permitted…” error:

if ($view == 'article' && $layout == 'edit' && !$this->checkEditId('com_content.edit.article', $id))
{
	// Somehow the person just went to the form - we don't allow that.
	$this->setError(JText::sprintf('JLIB_APPLICATION_ERROR_UNHELD_ID', $id));
	$this->setMessage($this->getError(), 'error');
	$this->setRedirect(JRoute::_('index.php?option=com_content&view=articles', false));

	return false;
}

After doing some quick debugging, we discovered that the problem with the above code was that the function checkEdit was returning false, for some unknown reason.

So we checked the function checkEdit which is located in the file legacy.php (which, in turn, is located in the folder libraries/legacy/controller), and we noticed that the following line...

$values = (array) $app->getUserState($context . '.id');

...was returning an empty array when it shouldn't. But why? And why is it the first time we try to edit the article it works, and the second time it doesn't?

And so we delved deeper and deeper and deeper into the session code while the wrinkles on our faces were multiplying by the minute (if you're a Joomla developer, you know that everything that has to do with sessions is a headache).

We then thought, we have an old copy of the same exact website residing on a development server. So we tested the old (development) website with a copy of the new database (e.g. we copied the database from the production website to the development website), and we didn't have the problem.

So it wasn't a database corruption causing this mess; it was a filesystem issue. We then compared all the changes in the PHP and the JavaScript files between the old website and the new website and we found no differences, except that the .htaccess file was present on the new website, but not on the old website which resides on the development sever.

So, in a moment of desperation, we copied the .htaccess from the production website to the development website, hoping that this will create the same issue on the development website, and it did! So the problem was with the .htaccess file.

Naturally, we started removing non-core lines from the .htaccess file one by one and then testing the website after removing each line, until we finally found the culprit, it was this line:

ExpiresDefault "now plus 1 hour"

The above line was instructing the browser to cache everything for one hour, including the actual PHP files, which was causing the session problems that we were seeing. What was weird is that it was only affecting Chrome.

So, how did we fix the problem?

Fixing the problem consisted of telling the browser not to cache PHP/HTML files, so, we added the following line to the .htaccess file (immediately after the aforementioned line):

ExpiresByType text/html "now"

(In case you're wondering, the above line instructs the browser to expire any PHP/HTML content immediately - in other words, the browser is instructed to always get that content from the server.)

We then tried opening an article, and then closing it, and then re-opening it, and it did re-open without redirecting back to the Article Manager page. Hooray! The problem was solved!

But why was the problem only occurring on Google Chrome?

We have no idea - and we didn't have the time to investigate why. It might be that either Chrome is very strict in listening to what the server tells it to do, or it might be that other browsers just don't listen that well (to the server), or it might be something else completely different. We don't know for sure.

If you're seeing the "You are not permitted to use that link to directly access that page", then check your .htaccess file, and make sure that you are not using any type of caching there. If you are, then we suggest you add the above line to your .htaccess file, it should work! If it doesn't, then please contact us. Our work is super quick, our fees are super competitive, and we know our Joomla!

How Content-Security-Policy Can Help Protect Your Joomla Website

Have you ever heard of Content-Security-Policy? If not, then we’re sure that you will find this post extremely informative and interesting…

In short, Content-Security-Policy is a policy set in the httpd.conf file, the .htaccess file, or as a meta tag (in the HTML code) that is typically used to prevent the website from running external scripts. In other words, it is excellent for XSS protection.

Let us explain a bit more…

You see, when someone visits a normal Joomla website, that has no Content-Security-Policy set, the browser loads all the images and scripts that are on the visited page, no questions asked. That’s what everyone wants right? Well, not really.

What every website administrator wants is to have the browser load all the images and the scripts from legitimate, approved sources, and not from just any source. What if, for example, the Joomla website that you are running has an XSS vulnerability (through a 3rd party extension) and was exploited by an attacker to request a hacked JavaScript file from a malicious source? If all the scripts are allowed to be executed, then an innocent end user will end up being redirected to a very bad website while potentially getting his PC infected with malware in the process. Of course, one could argue that the originating website is technically clean (e.g. the filesystem is clean), since all the malicious code is elsewhere, but that argument will not hold, simply because the end user only ended up on the bad website and with a virus on his computer because of the XSS exploited Joomla website. Additionally, not a single (respectable) online scanning tool and not a single search engine will think that it’s not the originating website’s fault.

This is where Content-Security-Policy comes to the rescue. When you have a policy for only allowing specific content from specific, trusted sources on your website, then you have come a long way in protecting and securing your website.

For example, adding the following line to your .htaccess file ensures that the only JavaScript and CSS files that are allowed to be downloaded from your website are those that actually exist on your website:

Header set Content-Security-Policy "default-src 'self'"

(Note: The location of the above line can be anywhere in the .htaccess file, but we like to add these things in the very beginning.)

Note that the above will not even allow inline scripts. For example, if you have some inline JavaScript code in your HTML, then that HTML code will be blocked, and you will see the following in Google Chrome‘s console:

Refused to execute inline script because it violates the following Content Security Policy directive: “default-src ‘self'”. Either the ‘unsafe-inline’ keyword, a hash (‘sha256-FU2Yz9Y7Q/i92m6ZTOAqpzhUeVAiTp1am3CtdegsQXs=’), or a nonce (‘nonce-…’) is required to enable inline execution. Note also that ‘script-src’ was not explicitly set, so ‘default-src’ is used as a fallback.

It will also will not allow inline CSS styles, and you will see the following message in Chrome‘s console if you have an inline CSS style:

Refused to apply inline style because it violates the following Content Security Policy directive: “default-src ‘self'”. Either the ‘unsafe-inline’ keyword, a hash (‘sha256-HhdiwPT6NvHjIA9I6BIJ1crwfwF/RLkZ6B/Ne8+ViGY=’), or a nonce (‘nonce-…’) is required to enable inline execution. Note also that ‘style-src’ was not explicitly set, so ‘default-src’ is used as a fallback.

Now we understand that the absolute majority of websites (including Google, Facebook, and Twitter) have inline CSS and inline JavaScript, so why is it blocked by default when Content-Security-Policy is enabled? Well, because it is unsafe. See, those large websites (Google et al) have many layers of protection and are not really worried of XSS attacks. But smaller, less secure websites which are managed by people who don’t know much about security and who use extensions written by complete strangers (who may or may not care about security) should worry about XSS, and should ensure that no scripts and styles exist in the HTML code.

But what about those websites that are sure of their security and that need to run inline scripts?

Well, if you’re sure about your website’s security and protection and you need to run inline scripts and inline styles, then change the above line to the following:

Header set Content-Security-Policy "default-src 'self' 'unsafe-inline'"

But what if you want to use scripts/CSS from other trusted domains such as externaltrusteddomain.com and you want inline scripts (but not inline CSS), for example. Well, you switch the above line with the one below:

Header set Content-Security-Policy "default-src 'self'; script-src 'self' http://*.externaltrusteddomain.com https://*.externaltrusteddomain.com 'unsafe-inline'"

What if you want to enforce the Content-Security-Policy server wide?

If you really want to do that, then all you need is to place the .htaccess line of your choice above in a VirtualHost tag of your httpd.conf file.

How can Content-Security-Policy be defined in a meta tag?

If you don’t have access to the .htaccess file, you can just modify the main index.php of your Joomla template and add the following meta tag (just after the opening head tag) to simulate the security policy of our last example above:

<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self' http://*.externaltrusteddomain.com https://*.externaltrusteddomain.com 'unsafe-inline'">

Which method is best for defining Content-Security-Policy?

We recommend you use the .htaccess file to store the content security policies, simply because it’s easy, it’s versatile, it’s clear, it’s part of the website (unlike the httpd.conf file which is part of the server, which means that migrating to a different server will cause you to lose these policies), and it is independent of the template (the meta tag option is template dependent, unless you use an extension to add it).

Is it a good idea to implement Content-Security-Policy directly on a production website?

Not really – Content-Security-Policy is a two edged sword. It will protect your website, but if you make a mistake in the rule, then your website will not function properly. And, because of its very confusing nature, it is very easy to make a mistake when writing such a rule (you may miss an external library, or an internal script, or an external font, etc…).

What we do is that we use the Content-Security-Policy-Report-Only directive instead of using Content-Security-Policy for testing prior to rule enforcement. The Content-Security-Policy-Report-Only is a directive that tells the system to silently send anything that doesn’t match the security policy to the URL of your choice as json encoded data in the body of the headers. The website will not be affected whatsoever when using this directive – with the exception that the browser’s console will display all policy violations.

For example, if you want to log everything that violates our last policy above, then you can add this line to your .htaccess file:

Header set Content-Security-Policy-Report-Only "default-src 'self'; script-src 'self' http://*.externaltrusteddomain.com https://*.externaltrusteddomain.com 'unsafe-inline'; report-uri http://www.[yourjoomlawebsite].com/parse-content-security-policy-report.php" 

Note that the only thing that we did was that we switched Content-Security-Policy with Content-Security-Policy-Report-Only and we added the URL that all policy violations should be sent to.

The parse-content-security-policy-report.php PHP file has one objective: to store the received data in the content-security-policy-report.json file. It should consist of the following PHP code:

$contentSecurityPolicyReportJSon = file_get_contents('php://input');
file_put_contents ('/home/[cpanel-user]/public_html/content-security-policy-report.json', $contentSecurityPolicyReportJSon, FILE_APPEND);

The saved json file can then be parsed using the json_decode PHP function. Keep in mind, that on high traffic websites, the json file can become very large very quickly. So, you should enable and disable this feature very quickly.

We hope that you found this post useful for protecting your Joomla website against XSS attacks. If you need help with the implementation, then you can always contact us. We’re always eager to help, we love our customers, and our fees are super duper affordable!

A Couple of Shell Commands to Find Hacked PHP Files on a Joomla Website

Note: This post assumes your website is running on a WHM environment. If your website is running on Plesk than the physical location of your Joomla website will be different.

Another note: This post assumes you have some very basic Linux knowledge. If that’s not the case then you can ask some Joomla security experts (such as, ahem, us) to help you.

Psst! Psst Psst! Psst Psst Psst! We have to tell you a secret – a secret that we have kept for ourselves for a long time for not-so-mysterious reasons! A secret that took us years to discover. A secret that we hope, once revealed, will make Joomla websites safer. Interested? We thought so…

What is that secret, we hear you ask? Well, the secret is that not a single alphanumeric string in Joomla’s core PHP files is larger than 62 characters, and not a single alphanumeric string with pluses and forward slashes is longer than 137 characters. OK – are you done laughing now? Let us explain…

You see, a huge sign of a malicious file is a long continuous string – which means that if we search for long continuous strings in the PHP files of a hacked Joomla website, then we will be able to find most (if not all) of the malicious/hacked files on that Joomla website.

Now the question is, how to find those files? Well, that’s easy, if you want to find all the PHP files containing alphanumeric strings that are more than 62 characters, then you will need to do the following:

  • Login to your Joomla website and clear your Joomla cache.
  • ssh to the server hosting your Joomla website.

  • cd to the directory where your Joomla website resides:

    cd /home/[cpanel-user]/public_html/

  • Run the following command to get the list of all the files containing strings that are longer than 62 alphanumeric characters:

    grep -r --include=*.php -e '[[:alnum:]]\{63,\}'

Now to find all the PHP files that contain alphanumeric characters including forward slashes and plus signs that are longer than 136 characters you just need to run the following command:

grep -r --include=*.php -e '[[:alnum:]\/\+]\{137,\}'

The above 2 commands are excellent for finding malicious/hacked files, and we use them when we are cleaning up a hacked Joomla website in order to weed out those files.

Will these commands really work?

Yes – and we use them every day to find those nasty malicious/hacked files on Joomla websites. The nice thing about these commands is that they are fast and they have a very high accuracy.

What about false positives? Will there be any?

If you have 3rd party Joomla extensions installed, then you may see some false positives, but they are easy to identify. For example, we know that Admin Tools and RSForm Pro will both yield false positives. There are other extensions of course that may have this issue – so you must carefully check each file returned by the above search before taking action (deleting it/cleaning it).

What about JavaScript files?

We’re glad you asked. You can use the same method for JavaScript files, except that the numbers are different: for core JavaScript files, the maximum size of an alphanumeric string is 149 characters, and that of an alphanumeric string with pluses and forward slashes is 480. Which means that we will need to run the below commands to get those potentially malicious JavaScript files:

grep -r --include=*.js -e '[[:alnum:]]\{150,\}'

and

grep -r --include=*.js -e '[[:alnum:]\/\+]\{481,\}'

Now the problem with JavaScript is that the maximums are quite high, so it’s possible to miss some malicious files because of that (maximums are inversely proportional to the accuracy). What we do is mainly use about half the numbers above (e.g. 75 and 240 instead of 150 and 481, respectively), and we ignore the core false positives.

What if the output of these commands is really large?

In the case of an overly hacked Joomla website (or one with many extensions installed), the above commands may return a lot of files, and that will most likely fill the scrollback buffer. To address this issue, you will need to pipe the results to a file (e.g. direct the generated output to a file instead of the screen). You can do this by adding a greater sign after the command followed by the name of the file. For example, the first command will be:

grep -r --include=*.php -e  '[[:alnum:]]\{63,\}' > potentially-malicious-php-files.php

That’s all for today! If you have some doubts on whether these commands will work for you – then try them, you won’t be disappointed. If you still think that there are some malicious files on your server, then please contact us. We are experts in Joomla security, our fees are super affordable, and we will clean and secure your website!

How to Completely Disable Session Activity for Visitors on Joomla Websites

Note: This post contains core modifications. If you’re not comfortable with doing core modifications, then please ask some Joomla experts to do it for you.

Another note: This post assumes that your website doesn’t allow users to login to the frontend – if that’s not the case then please be aware that the modifications below must be modified (What? The modifications need to be modified? What kind of English is this?) to accommodate your needs. If you think this is a bit over your head, then we suggest you contact us!

Joomla has the nasty habit of creating and maintaining a session in Joomla for every visitor, which is an unnecessary overhead that causes performance issues on high traffic websites.

Luckily, at itoctopus, we have a solution for everything Joomla: in a previous post, we have discussed how to minimize write activities to on the #__session table, in this post, we will explain how to completely disable write activity for visitors on the Joomla website.

So, how can you completely, fully, unequivocally (can this word be used in this context?) disable all activities on the #__session table for visitors on a Joomla website?

Well, you can do that by following the below guide:

  • Open the file cms.php which is located under the libraries/cms/application/ folder.
  • At the very beginning of the checkSession function, add the following code:

    if (stripos($_SERVER['PHP_SELF'], '/administrator/') !== 0)
    return;

    The above will disable initial creation of the session for all pages with the exception of administration pages (including the login page for administrators).

  • Open the file database.php which is located under the libraries/joomla/session/storage/ folder, and add the following code at the very beginning of the read, the write, the destroy, and the gc functions:

    if (stripos($_SERVER['PHP_SELF'], '/administrator/') !== 0)
            return;

  • That’s it!

After adding the above code, your Joomla website will no longer do any session activity for your visitors; it will not insert, update, retrieve, or delete anything from the #__session when someone is visiting your website. This will lessen the load on your server (the load reduction will be noticeable on high traffic websites) and will reduce the impact of DoS attacks.

We know that some of you might be intimidated by the thought of updating core files, so, if you need help with that, please contact us. Our rates are super affordable, our work is super clean, and we are super friendly!

The Hidden Downside of Embedding Joomla Modules in Articles

One of the really nice features that Joomla has is loading modules within articles using loadmodule or loadposition (or using an 3rd party extension such as Modules Anywhere). It is an excellent functionality that permits Joomla administrators to leverage the power of modules within articles (or even within modules if using Modules Anywhere).

That flexibility, however, comes at a cost: embedded modules are only cached when the System – Page Cache plugin is enabled (and we all know how fun it is to have this particular plugin enabled).

This means that if you have some heavy queries running in those embedded modules, then expect those heavy queries to run every time even if you are using conservative or progressive caching (see this post for the difference between the two). Yes – that’s not optimal, and unfortunately, there are no workarounds that don’t involve substantial modifications to the core.

How did we discover this?

Well, we discovered this whole thing by accident (just like when Alexandar Graham Bell discovered the phone, and just like when Christopher Columbus discovered America): we were analyzing the queries that a certain website was sending to the database server in a silo’d environment, and we noticed that all the queries loading the modules were not being executed when either the conservative or the progressive caching, with the exception of the embedded modules.

You can try this yourself:

  • Copy your website to a development server.
  • Enable Conservative Caching in your global configurations settings.

  • Add the following lines to the my.cnf file (the my.cnf file is the MySQL configuration file, which is typically located under the /etc folder) in order to catch all the queries that are sent to the database server (you can use vi or nano to edit the file – we prefer vi):

    general_log = on
    general_log_file=/var/lib/mysql/all-queries.log

  • Restart MySQL by issuing the following shell command:

    service mysql restart

  • Visit the website using your browser, and then clear the all-queries.log file by issuing the following command:

    > /var/lib/mysql/all-queries.log

  • Visit the website again, and you will see that all the queries pertaining to those embedded modules are still being sent to the MySQL database server and recorded in the all-queries.txt file.

So yes, while embedding module is a very powerful concept, it also comes at a price. For some modules and some websites, that price is very expensive in terms of performance. On the bright side, however, for most websites and most modules, the price for doing that is almost negligible.

If your website has a lot of embedded module and you are noticing performance issues caused by these modules, then just contact us. Our prices are affordable, our work is professional, and we always have a solution for anything Joomla!

A Super Quick Guide to Deal with a Hacked Joomla Website

You wake up one morning, and you get that nasty email from Google telling you that your Joomla website is hacked. So what do you do? Well, first of all you relax knowing that this can happen to anyone in this galaxy, and then, you follow the below guide:

  • Backup your Joomla website (filesystem and database) so you can revert back to a previous backup in case of a major mistake during the cleanup process.
  • Delete all the files under the cache and the tmp folders.

  • Check your .htaccess file(s) for anything malicious. Note that we are using file(s) instead of file since you may have more than one .htaccess file (for example, one under the root directory of your Joomla website, one under the administrator folder, and one in a directory even higher than that of the Joomla root directory [this is typically the case when the Joomla website is operating as an add-on domain or is hosted on a toxic shared hosting environment]).

  • Overwrite your Joomla files with a fresh Joomla install as described here. In short, you will need to find which version of Joomla you are running, download that exact version of Joomla from the official website, remove the installation and the images folder from the downloaded zipped file, and then extract the zipped file onto your website.

  • Run a malware scan (such as maldet) to see if there are still some malicious files. If the scan finds any files, then clean them or delete them (depending on whether these files are actually part of your Joomla website or not). Ensure that your malware scanner is up-to-date before running the scan.

  • Make sure that the index.php file is the only file that can be accessed by the outside world. You can do this with an .htaccess rule – we have explained how to do this in a previous post.

  • Use an external scanner to check if your website is clean:

    • If it’s not clean, then try disabling modules/plugins one by one and then repeat the scan each time you disable a module/plugin until the problem is no more. Stop when the website is reported as “clean”, and then examine closely the module/plugin that you have just disabled as it is the one that is hacked and that is poisoning the whole website.
    • If still not clean, then switch to a different template and repeat the scan and see if the problem is fixed. If the problem is fixed, then the hack is somewhere in your template. Either uninstall your template and re-install it or fix the hack manually.

    • If not already done, update your Joomla website to the latest version.

    • Update all the extensions (template, modules, plugins) on your Joomla website to the latest version. This step and the previous step are absolute musts for website protection.

Typically, following the above guide will get your website clean and more secure. If it doesn’t, or if you need help with this guide, then please contact us. Our fees are affordable, our work is top notch, and we will clean your website.

Internal Server Error on Joomla’s Backend Login Page – How to Fix

A client emailed us very early in the morning and told us that whenever he tried to access the login page of the backend of his Joomla website (e.g. http://www.ourclientjoomlawebsite.com/administrator ), he was seeing a 500 Internal Server Error page. He told us that the frontend worked flawlessly, and the problem was only in the backend.

In most cases, those Internal Server Errors, when they are restricted to the backend, only appear after the person tries to log in, and not before, which made this whole issue a bit curious.

Naturally, when something happens on the very first page, we rename the .haccess to htaccess.old, and we did that, but it didn’t help. But then we noticed that there was another .htaccess located under the administrator folder, so we renamed it to htaccess.old, and then we tried to login, and this time, it worked! We saw the beautiful Joomla login page waiting for us to enter the credentials. We logged in with the credentials provided by the client, and were able to see the backend!

But why was the .htaccess file causing this problem?

The client apparently just moved his website from one server to another, and had the following in the .htaccess file (that is located under the administrator folder):

AuthName "Authorisation Required"
AuthUserFile /home/.htpasswd
AuthType Basic
Require valid-user
ErrorDocument 401 "Authorisation Required"

Apparently, he was using our method for an additional authentication layer using the .htpasswd/.htaccess combination. However, when he moved his website to a new server, he didn’t copy back the corresponding .htpasswd file to the /home folder, which was the cause of the problem. We informed him about the cause of the issue and so he copied back the file from his old server to his new server, reinstated the .htaccess file, and the problem was no more.

Now if you, our dear reader, are seeing an Internal Server Error when you just browse to the login page of your Joomla backend, then the first thing that you need to do is to rename your .htaccess file (under the root of your Joomla website, and then under the administrator folder). If that still didn’t work, then please contact us. We will fix the problem for you quickly, efficiently, and for very little money.

Minimizing Brute Force Attacks on Joomla’s Backend Using .htaccess

If you’re running a Joomla website and you regularly check your Apache web server logs, you will notice that these logs are full of brute force attacks. These brute force attacks consist of continuous POST requests to your Joomla website, with dictionary based combinations of usernames and passwords, with the hope that one of these combinations will be the right one. Obviously, unless you have a very obvious password, brute force attacks are just a nuisance and nothing more. However, in large numbers, brute force attacks are no longer nuisances, but a major issue, because of the load issues they may cause.

Luckily, blocking brute force attacks on the administration area of a Joomla website is easy. In fact, just last month, we published a post with a ModSecurity rule to block brute force attacks on the backend of a Joomla website.

If you’re not a big fan of ModSecurity, then you can always add an additional layer of authentication on the backend of your Joomla website using the .htpasswd.

The nice thing about the ModSecurity and the .htpasswd methods is that they eventually block the offending IP – but – on the flip side, they are not very easy to implement.

If you want an easier method to minimize brute force attacks on Joomla websites, then you’re in for a treat! Today, we have an easy set of rules that you can simply add to your .htaccess file to minimize such attacks. Here they are (we know you don’t want to wait anymore):

# First redirect from non-www to www as explained here.
RewriteEngine On
RewriteCond %{HTTP_HOST} ^yourjoomlawebsite.com
RewriteRule (.*) http://www.yourjoomlawebsite.com/$1 [R=301,L]

# Now block brute force logins
# If the request type is POST...
RewriteCond %{REQUEST_METHOD} =POST
# ... and the referering page is not your website
RewriteCond %{HTTP_REFERER} !^http://www.yourjoomlawebsite.com [NC]
# then block the request
RewriteRule ^(.*)$ - [R=403,L]

The above code should be added to the .htaccess file under your administrator folder. If you don’t have one (which is typically the case for most Joomla websites), then create it and add the above code.

Now let’s explain the rules a bit (we did explain them in the code, but more explanation is never harmful):

  • First we redirect from non-www to www to make things easier.
  • We then check if the request method is of type POST – if it is, then we check if the referring URL is not the actual website (brute force attacks typically post directly to the action page without going through the normal workflow). If it is not, then we block the request with a 403 forbidden. If it is, then we allow the request.

But is this too good to be true?

We hate to say it – but yes – this whole thing is too good to be true, and that’s why we carefully chose the word minimizing instead of blocking in the post’s title. See, the heart of the .htaccess rules that we have is the HTTP_REFERER value, which, unfortunately, can be forged because it is a client set value (e.g. it is set by the client, and not by the server).

Additionally, there are some browser plugins that can automate tasks, including brute force attacks, and these browser plugins make it look like as if the whole thing was initiate by a real user, and not by an automated robot.

But, again, these rules will minimize the brute force attacks – since many attackers don’t even both faking the HTTP_REFERER, so implementing the above code is beneficial to your Joomla website!

Now, if you have some questions on the above, or you need help with implementation, then please contact us. Our fees are affordable, our work is quick, and we really are experts in Joomla security.

MySQL’s “SELECT INTO OUTFILE” – Should Joomla Administrators Be Afraid?

Note: This post is very advanced and is targeted at system administrators and advanced programmers.

Another note: The aim of this post is strictly to promote security on Joomla websites and to investigate potential threats. It is not aimed at teaching others how to exploit websites.

After reading a post this morning on the dangers of the “SELECT INTO OUTFILE” MySQL query statement, we initiated an investigation mission so that we can validate whether its threats are real or not.

Now, of course, you might be wondering, what is “SELECT INTO OUTFILE”?

“SELECT INTO OUTFILE” is a MySQL query statement that will write the selected rows into a file. For example, if you want to write the contents of the #__content table into a file, then you can run the following query:

SELECT * INTO OUTFILE '/tmp/content.csv' FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\n' FROM `#__content`;

Theoretically, the above query should dump the content of the #__content table into the content.csv file which is located under the /tmp folder, which is fine, but let’s look at another example:

SELECT "" INTO OUTFILE '/home/[cpanel-user]/public_html/outfile.php'

The above query will (theoretically – you’ll see later in this post why we’re using the word “theoretically” a lot) generate a PHP file called outfile.php under the main directory of the Joomla website. Since the file is typically created with rw-rw-rw permissions, then this means that it can be accessed from the browser by going to: http://www.yourjoomlawebsite.com/outfile.php.

The outfile.php that we created is harmless, but one doesn’t need to have the imaginative mind of Charles Lutwidge Dodgson (aka Lewis Carroll, the man who wrote Alice in Wonderland, who sometimes believed in as many as six impossible things before breakfast), to know what a malicious user can do with that powerful command.

In order to know how much of a threat that query was, we created a test environment and played the Exploit my test website using “SELECT INTO OUTFILE” game, which rules were the following:

  1. We had to upload a PHP file to the root directory of the website using the “SELECT INTO OUTFILE” query.
  2. There are no other rules, which means that we can do anything on the server and change anything to succeed, including using our root privileges to the test server.

Here’s how we started playing the game:

  • We opened the file index.php which is located under the main Joomla directory.
  • We added the following code to its very end:

    $db = JFactory::getDbo();
    $sql = 'SELECT "<?php echo(\'Hello World from MySQL\'); ?>" INTO OUTFILE \'/home/[cpanel-user]/public_html/outfile.php\'';
    $db->setQuery($sql);
    print_r($db->query());

When we tried to load the website, we saw the following error:

1045 – Access denied for user ‘[mysql-database-user]‘@’localhost’ (using password: YES) SQL=SELECT “<?php echo(‘Hello World from MySQL’); ?>” INTO OUTFILE ‘/home/[cpanel-user]/public_html/outfile.php’

A quick investigation revealed that cPanel database users are never given the FILE privilege, which is required to execute “SELECT INTO OUTFILE” queries. In fact, even if you grant all privileges to the database user from within the cPanel interface, the FILE privilege will not be granted. It has to be granted manually by the root user (using the GRANT FILE ON [joomla-database].* TO [mysql-database-user] query)…

Speaking of the root user, it already has the FILE privilege, so why not use it since we are in a consequence-free environment? So, we changed the database user in the configuration.php file to root (we also changed the password to that of the root), and we tried again, and here’s what we saw this time:

1 – Can’t create/write to file ‘/home/[cpanel-user]/public_html/outfile.php’ (Errcode: 13 – Permission denied) SQL=SELECT “<?php echo(‘Hello World from MySQL’); ?>” INTO OUTFILE ‘/home/[cpanel-user]/public_html/outfile.php’

Aha! So, the root user was almost able to write the file, except that the file permissions prevented him from doing so. So, we changed the permissions of the public_html folder from 755 to 777 (775 didn’t work) and we tried again, and, unsurprisingly, it worked this time and it created the file. Wow!

Now, we wanted to verify that we were able to execute the file from the browser, so we pointed our browser to: http://www.testjoomlaenvironment.com/outfile.php, thinking that we will see the Hello World message, but instead, we saw this error:

Forbidden – You don’t have permission to access /outfile.php on this server.

Then we remembered that we had the following security rules in the .htaccess file to only allow the index.php file to be executed:

<Files *.php>
    Order Allow,Deny
    Deny from all
</Files>
<Files index.php>
    Order Allow,Deny
    Allow from all
</Files>

Removing those rules allowed the execution of the PHP file, and displayed the Hello World from MySQL message, which automatically made us win the game. Hooray!

But was it a fair game?

We admit, the game we played was not a fair game. First, there is a major assumption in this game, and it is that there is an exploit somewhere on the Joomla website that will allow us to inject the code, which, realistically, is only the case on a small percentage of Joomla websites. Second, even if there was an exploit, if it wasn’t for the second rule, we would have never been able to win; we would have been stuck trying to make the initial database user execute the “SELECT INTO OUTFILE” command.

But what really made the game impossible to win without “cheating” (e.g. without the second rule) was the fact that the WHM/cPanel environment is a very secure environment. For example, the privileges granted to database users do not include the FILE privilege and the permissions on the folders are correct (compare that to a Plesk environment where the whole thing looks like a mess [Note: We are not hinting here that Plesk, as an environment, is insecure or less secure than WHM but we have yet to see permissions done correctly on a Plesk hosted website]).

As you can see, the “SELECT INTO OUTFILE” query is a dangerous query, but you are more likely to win the lottery than get hacked using this query if you’re on a WHM/cPanel environment. If you’re on a less secure environment, then it is very important to make sure that the database user that you’re using does not have any FILE privilege. If in doubt, please contact us. We are always ready to help, our fees are super affordable, and we know how to ironclad a Joomla website.

MySQL InnoDB Optimization for Large Joomla Websites

One of the first things that we used to do to resolve a substantial part of load issues caused by a Joomla website was to switch the main content tables from MySQL InnoDB database engine to MySQL MyISAM. The results were always impressive. Nevertheless, we always had conflicting feelings after the switch: while we were happy that the load was down, we knew that we were playing with fire and we knew that there’s a way to make InnoDB work on a very large Joomla database, but we decided to play re-actively on this issue, and not proactively.

The reason why we knew that we were playing with fire is the fact that MyISAM locks the whole table when inserting, deleting, or updating rows, which means that any subsequent CRUD (Create, Read/Retrieve, Update, Delete) activity will have to wait for the lock to be released to get executed. Of course, the locking time is typically very small, but things can get very ugly very quickly when a massive update to a critical table takes place, and so all the queries needing that table will have to wait for a long time, creating an unmanageable backlog of queries, ultimately causing the server load (caused by MySQL) to skyrocket.

The nice thing about InnoDB is that, unlike MyISAM, it only locks the row(s) it is updating/adding/deleting, essentially reducing any lock time to nil. But, for the life of us, we were unable to get InnoDB play well on large Joomla websites powered by large databases.

A few weeks ago, however, everything changed. We started seeing some minor performance issues on a large Joomla website that we manage. We ssh’d to the server and we noticed that the load was hovering around 3, which is not bad, but not good either. A load of around 3, when the main load is caused by MySQL (and not by other processes), typically hints that there are some slowdowns on the website. So, we optimized the website even more (we performed many core optimizations on that particular website before), but the outcome wasn’t great. We were frustrated, and we knew that we have reached the physical limit of MyISAM – but we were just afraid of jumping into the abyss.

Eventually, we decided to take that leap of faith, because, as mentioned in the beginning of this post, we knew that InnoDB would perform much much better than MyISAM provided the right database configuration settings were chosen. We were so determined to making this whole thing a success that we even switched to a better server that had double the memory and a faster SSD. Here are the server details (note that the website in question receives 50,000 uniques/day, and, believe it or not, the System – Cache plugin is disabled on that website):

Processor: Intel Xeon E5-2650 v3 10 Cores
Memory: 64GB DDR4 SDRAM 0.00
Primary Hard Drive: 6 x 500GB Crucial SSD Hardware RAID 10
Secondary/Backup Hard Drive: 1TB SSD Backup Drive 0.00
Linux OS: CentOs 6 - 64Bit 0.00
Control Panel: cPanel / WHM - Fully Managed

Since we were switching to InnoDB, we cared mostly about 3 things: more RAM, a fast SSD drive, and a fast processor (in that order). The RAM was doubled (the client had 32 GB before), the SSD drive was slightly faster, and the processor was much faster (despite the fact that the client had a 48 core AMD processor before).

After moving to the new server, we noticed that not much has changed, the load dropped slightly and hovered around 2.8 (which was still not good) and the (unfixable) slow queries were still being recorded to the slow query log. It was definitely a good time to test the power of InnoDB to see whether it solves the problem or not.

How we switched from MyISAM to InnoDB

Switching from MyISAM to InnoDB consisted of 3 steps:

  1. Modifying the MySQL configuration file to ensure InnoDB runs optimally.
  2. Restarting the MySQL server.

  3. Switching all the database tables from MyISAM to InnoDB.

Of course, the most difficult part was finding the right InnoDB configuration settings for that particular website, and after a lot of very hard work, we found them! Here they are:


innodb_buffer_pool_size = 8G
innodb_log_file_size = 2G
innodb_log_buffer_size = 256M
innodb_flush_log_at_trx_commit = 0
innodb_flush_method = O_DIRECT

After determining the above values, we added them to the my.cnf file in the /etc folder.

Of course, the million dollar question (if you’re reading this post in the year 2100 then please change million to billion for accuracy) is, how to determine the above values? While we know that you are not going to give us a million dollars for our answer (or is it that the one who asks the question gets the million dollar? We’re not really sure!) we are going to explain how we got the above values for you anyway:

innodb_buffer_pool_size is probably the most important value here. InnoDB loads all the tables in the RAM, and so when a query hits an InnoDB table, the database engine will check if that table is already in the RAM, if it is, then it queries the table from the RAM (and not from the disk), if it isn’t, then it loads it in the RAM, and then queries it from there. innodb_buffer_pool_size, as you may have guessed, is the maximum amount of RAM that can be used by InnoDB for table storage – if InnoDB reaches that maximum, then it’ll start querying tables directly from the disk, which is a much slower process. Most references state that the amount of RAM allocated to innodb_buffer_pool_size should be 80% of the total RAM, but we think this is just too much. Additionally, these references assume that the database server runs on a different physical server, which is not the case for the absolute majority of our clients (they have the database server, the web server, and the mail server, all running on the same physical server). Since innodb_buffer_pool_size will contain, at max, all the data in all the databases, then an intelligent way to determine its value would be by calculating the size of the /var/lib/mysql folder by issuing the following command in the shell:

du /var/lib/mysql -ch | grep total

In our case, the above returned 2.4 G, so we just multiplied it by 3 and we rounded it, and we came up with 8 G (8 Gigabytes). Of course, even 3 G could have worked for us, but we just didn’t want to take any chances and we had a lot of RAM. Don’t be frugal with this value as it’s the most critical in an InnoDB environment.

innodb_log_file_size is the size of the physical logs where all the recent database activities are cached for both performance reasons and to restore the database after a crash. innodb_log_file_size is a bit more complex that innodb_buffer_pool_size to accurately determine (make that highly complex). But, a good rule of thumb that works well for most database instances it to set this value to 25% of the innodb_buffer_pool_size, which is in our case 2 G.

Of course, you might be asking yourself, why not just set the innodb_log_file_size to a very high number and then forget about it? Well, you shouldn’t do that because of 2 reasons:

  1. Restarting MySQL will take a very long time. In fact, when we restarted MySQL with innodb_log_file_size changed to 2 G, we had to wait for about 3 minutes (when the value was set at about 256M, restarting the server took just a few seconds), this is because MySQL had to write 2 files of 2 G each to the fileystem: ib_logfile0 and ib_logfile1 (now you know what these weird ib_logfiles files under the /var/lib/mysql folder are).
  2. Restoring a crashed database will also take a lot of time. Keep in mind that a crashed database is not a remote possibility, and we’ve seen it happen many times (almost all cases were caused by a power outage).

innodb_log_buffer_size is the size of the RAM that InnoDB can write the transactions to before committing them. Honestly, a value of 64 M is more than enough for almost any MySQL database server (most MySQL servers have it set to 8 M), but we changed it to 256 M when experimenting and we didn’t change it back as a large innodb_log_buffer_size has zero negative effect on the server. In all fairness, probably a value of 1 M could have worked for us because we’re not even using database transactions anywhere on the Joomla website (see below).

innodb_flush_log_at_trx_commit is (more or less) an on/off setting for database transactions (when set to 1, it means we are allowing database transactions, when set to 0, it means we are not allowing database transactions). The absolute majority of Joomla websites out there don’t use database transactions (for clarification, a database transaction is completely different from a payment transaction), which means that this should be set to zero (0). The default value of 1 means that the InnoDB database engine is ACID (Atomicity, Consistency, Isolation, Durability) compliant (meaning that the transactions are atomic), which creates a significant overhead on the MySQL database server (even when it’s not needed).

innodb_flush_method is a setting to tell MySQL which method to use to flush (delete) the database data (not actual data, but the cached data) and logs. If you’re using RAID, then set it to O_DIRECT. Otherwise, leave it empty.

After applying the above settings in the my.cnf file, restarting the MySQL server, and then changing all the tables from MyISAM to InnoDB, we saw a huge improvement in the performance: the load dropped to slightly over 1, and the slow query log stopped recording slow queries every few minutes (in fact, we only had 3 slow queries since we did the switch 3 weeks ago). Even Google saw a substantial improvement, see for yourself:

Time spent downloading pages: Before and after InnoDB Switch

Figure 1: Time spent downloading pages: Before and after InnoDB Switch (this is a snapshot taken from the “Crawl Stats” page from Google Search Console)

As you can see, the time spent downloading a page has decreased significantly as of June 30th (which is when we switched to InnoDB). One can safely say that the download time was halved! Impressive, huh?

We hope that you found this post useful. If you have need help with the implementation, then you can always contact us. Our rates are super affordable, our work is super clean, and we will make your Joomla website much, much faster!

“The File Cache Storage is not supported on this platform.” Error After Updating to Joomla 3.6

A client called us very early this morning and told us that after he updated to Joomla 3.6 (from Joomla 3.5.1), he was seeing the following fatal error on the Joomla backend:

The File Cache Storage is not supported on this platform.

We immediately investigated the issue (it was around 5 AM), and we realized that it was caused by the following code in the storage.php which is located under the libraries/joomla/cache folder:

// Validate the cache storage is supported on this platform
if (!$class::isSupported())
{
	throw new RuntimeException(sprintf('The %s Cache Storage is not supported on this platform.', $handler));
}

Tracing the isSupported function, we found out that it is located (for that particular scenario) in the file.php file which, in turn, is located under the libraries/joomla/cache/storage folder. Here is its code:

public static function isSupported()
{
	return is_writable(JFactory::getConfig()->get('cache_path', JPATH_CACHE));
}

So cache_path, which is (almost always) JPATH_CACHE, which translates to the cache folder or to the administrator/cache folder (depending on whether the user is browsing the frontend or the backend) is not writable by Joomla (e.g. it is not writable by the web server, which was Apache in the case of our client).

Since the issue was only happening on the backend, then this meant that the folder administrator/cache was not writable. So, in order to fix the problem, all that we had to do was to change the permissions and the ownership on that folder to make it writable by Joomla.

Why was the problem only happening on the backend?

The problem was only happening on the backend because the cache folder, located under the root directory of the Joomla website, was writable, but the one under the administrator folder wasn’t.

Why is this suddenly an issue with Joomla 3.6?

Previous versions of Joomla did not have the RuntimeException statement when the cache folder wasn’t writable. They were silently handing the issue by just not caching the content, which is the right way to do things. We think that this problem will cause the Joomla core developers to act and revert that change.

In any case, if you’re having this error on your Joomla website, then make sure that you have both the cache and the administrator/cache folders writable by your web server. If you need any help, then please contact us. Our fees are super affordable, our work is fast and professional, and we are the friendliest Joomla developers on planet Earth!

Searching for the “*/$” Pattern on Your Joomla Website to Discover Hacked Files

Over the course of the last decade, we have unhacked and secured many Joomla websites ranging in size from very small to very large. In most cases, cleaning up a Joomla website consists of the following steps:

  1. Blocking all traffic to the website.
  2. Backing up the website.

  3. Overwriting the core files from a fresh Joomla install matching the same version.

  4. Scanning the website using various tools and deleting the malicious files.

  5. Only allowing the index.php file to be executed by Apache as described here.

  6. Preventing direct access to core folders.

  7. Installing the itoctopus Joomla firewall (a homemade firewall that we install for our clients).

  8. Updating the Joomla website to the latest stable version and updating/uninstalling vulnerable extensions (optional – as some clients want to only cleanup the website and prefer to update at a later time [of course, we always recommend to perform both the cleanup and the update at the same time]).

  9. Testing the website (we whitelist our IP in order to do that).

  10. Backing up the clean version of the website.

  11. Unblocking access to the website.

Typically, the various tools that we use in step number 4 above find all the malicious files on the website, but not always, which was a major problem and which caused the cleanup process to take a lot more time.

However, lately we have noticed the following pattern in almost all the malicious files that were not caught by the scan: */$

Let us explain…

You see, malicious code is typically highly compressed, and so usually, there are no spaces between a comment and a variable declaration, which means that searching for */$ will most likely return all of the malicious files on your Joomla website. Here’s the ssh command that you should use to search for it:

grep -R --include=*.php '\*\/\$' *

(Note: This command should be run at the root directory of your Joomla website, e.g. at the same level of the main index.php file).

The above command may return some false positives, but that’s OK, because they are typically few, and are easily recognizable.

We hope that you found this post useful and that it helped you cleanup your Joomla website. If you still think that your website is not fully clean, or if you want some Joomla security experts to do the cleanup for you, then please contact us. We will clean your website, we will secure it, and we will not charge you much!

“Could not connect to MySQL.” Error on a Joomla Website

Yesterday (Friday) we explained how a full disk brought the Joomla website of a client of ours completely down. We also mentioned that we upgraded the VPS plan of that client to a more generous plan in terms of disk drive and memory – what we didn’t mention, however, was that the VPS upgrade did not go smoothly, because when the upgrade was done, we saw the following error on the homepage of the website:

Error displaying the error page: Application Instantiation Error: Could not connect to MySQL.

Now, probably most of those who have an intermediate knowledge of Joomla know that the above error is caused by one of the following:

  • The database username and/or password and/or database name and/or database host is/are wrong.
  • The database user does not have full read/write access to the database.

  • The database server is not running.

  • The mysqli PHP module is not installed.

We were confident that the error was not caused by any of the above, nevertheless, we decided to create a small PHP file that will test the connection to the database just to be sure. So, we created a file called check-mysql-connection.php which contained the below code:

<?php
$dbConnection = new mysqli("localhost", "[db-username]", "[db-password]", "[db-database-name]");
if ($dbConnection ->connect_errno !== FALSE)
	echo $dbConnection ->connect_error.' (Error Number: '.$dbConnection ->connect_errno.')';

?>

Running the above file generated the following error:

Warning: mysqli::mysqli(): (HY000/2002): No such file or directory in /home/[cpanel-user]/public_html/check-mysql-connection.php on line 2
No such file or directory (Error Number: 2002)

Hmmm! So it was something with mysqli. We immediately assumed that it wasn’t installed on the new VPS instance, so we logged in to WHM, went to EasyApache 3, and then checked the ** DEFAULT ** (selected) profile. We noticed 2 things:

  • The mysqli module was installed.
  • The VPS instance was running an EOL (End Of Life) version of PHP, which was version 5.4.45.

So we switched, on the PHP Version page (in EasyApache), to PHP 5.5.37. We then clicked on Next Step, and then we clicked on Save and Build. We followed the instructions and then tested the website, and this time it worked!

But why didn’t the mysqli PHP module work with PHP 5.4.45?

We suspect that was because the installed mysqli module wasn’t compatible with that version of PHP. Upgrading to PHP 5.5.37 ensured that everything was compatible with that particular version of PHP.

So, the next time you see that Could not connect to MySQL error on your Joomla website, make sure (after verifying the obvious: username, password, host, permissions, etc…) that the problem is not with your version of PHP. If you need help finding and fixing the problem, then that’s what we’re here for. Just contact us and we’ll solve the problem for you quickly, professionally, and for a very small fee!

“An error has occurred” When Trying to Login to the Backend of a Joomla Website

A regular client of ours called us a couple of hours ago and told us that her company’s website was down, and she was seeing a weird error when trying to login to the administrator section of her Joomla website.

We immediately checked the website and we noticed that the website was complaining about too many redirects. So we asked the client whether she did any redirects lately, and she told us that she did one yesterday through sh404SEF, but the website was working fine until this morning.

So, we tried to login to the website, but we saw the following error as soon as we went to http://www.ourclientjoomlawebsite.com/administrator/:

An error has occurred.
404 Not Found

Yes – we were getting a 404 Not Found when trying to access the login form (before even trying to login), even though the index.php file under the administrator folder had the right permissions and was being loaded by Joomla (we added a die statement at its very beginning to make sure that it was indeed being used).

At this point, we started suspecting that the website was hacked. Since we were confident that that particular website had no core modifications, we thought about applying our super quick cleanup method by just uploading a fresh copy of Joomla (matching the version of our client’s Joomla website) through cPanel and extracting it onto the existing Joomla website. But, as soon as we tried to login to the cPanel account of our client, we saw the following error:

Sorry for the inconvenience!

The filesystem mounted at / on this server is running out of disk space. cPanel operations have been temporarily suspended to prevent something bad from happening. Please ask your system admin to remove any files not in use on that partition.

Aha! The system completely ran out of space, and this explained why the website was down: both logging in to the administrator and viewing the homepage required write activities to the database/filesystem that were not allowed!

Just to make sure, we ssh’d to the server, and we issued the following command…

df -m

…and we noticed that /dev/vda3 (the main website partition) was 100% used.

So, how did we fix the problem?

We fixed the problem by removing old backups from the server, which reduced usage on the aforementioned partition to about 70%. Once that was done, we deleted the content of the cache folder under the website’s directory and we checked the website. And Voilà, everything was working again!

Once we were sure that the website was stable, we upgraded the client’s VPS to a higher plan that guaranteed double the disk space (and double the RAM).

If you’re seeing a 404 error when trying to access the backend of your Joomla website, then do not panic: your website is still there. It’s just that your server ran out of disk space, and you will have to remove unnecessary backups for everything to work again. If you want us to help you resolve this problem, then please contact us. Our prices are right, our work is professional, and we are very very reliable!

An Awk Line to Generate a List of 404 Pages Sorted By Number of Hits

Note: This post assumes a WHM based environment. If you are using Plesk or anything else, then the log path may be different. Please take that into consideration.

With time, most websites will have an increasing number of 404 pages. This is natural and it is caused by changes in the link structure, wrong incoming links, deleted/moved files, etc… The majority of websites typically ignore 404s, but we know that this is not the best route to take, especially in Joomla: in a previous post, we have explained the dramatic load issues that 404 pages can cause on Joomla websites. Now while we did propose, in that post, a core solution to address the consequences of 404 pages (which is also critical to lessen the effects of DoS attacks), that doesn’t mean that we think that 404 pages should be left unattended. In fact, we think that, as a Joomla administrator, you should analyze your logs often (ideally once a month), and, at the very least, address the 404 pages that appear the most in your logs to ensure long term stability of your website and your server (not to mention the positive SEO implications of doing so).

To make that task easier, we have devised a line, just one simple line, which will list all the 404 pages that you have in your logs reversely ordered by the number of hits. This will allow you to address the 404s that matter most. The line consists of a shell command using awk, a very powerful Linux shell tool mainly used for searching and analyzing files. Without further delay, here is the line (note that you must ssh as root in order to run the below command):

awk '{ if ($9 == 404) print $7 }' /home/domlogs/cpanel-user/yourjoomlawebsite.com | sort | uniq -c | sort -nr > 404-urls.txt

(Note: cpanel-user is the cPanel account username of your domain. Yes, duh!)

The above command checks each line (in the log file) if its 9th field is 404, if it is, then it will extract the URL from that line (which is the 7th field), put it in temporary memory, and then get a list of all the unique URLs as well as the number of their occurrences in the original log file. Finally, it sorts the list of unique URLs based on the number of occurrences of each URL, and then outputs the result to the file 404-urls.txt.

But why not just use Google Search Console to get the list of 404 pages?

Google Search Console (previously known as Google Webmaster Tools) generates a list of 404 pages on any website. However, that list is often out of date and does not generate the accurate results that one gets when examining the Apache access logs. Nevertheless, it is still a good idea to address all the issues that Google Search Console finds on your website, including 404 pages.

We hope that you found this post useful. If it didn’t work for you, then it might be that your log paths are different on your server. If you can’t find them or if you need help with the implementation, then please contact us. Our work is top notch, our fees are affordable, and we are the most courteous and friendliest Joomla developers on planet Earth.

No Save Button in Joomla’s Backend – How to Fix

A client emailed us this Saturday morning and told us that he’s not able to see the Save button when he opens any content item for editing (such as articles, categories, modules, etc…). So we logged in to the backend of his website and we noticed that not only the Save button was missing, but also the following buttons: Save & Close, Save & New, Save as Copy, Versions, and Close. In fact, the whole toolbar was missing.

We immediately knew what the problem was: the Toolbar administrator module was disabled. This meant that all we needed to fix the problem was to re-enable the module, and we did it the following way:

  • We went to Extensions -> Modules in the Joomla backend.
  • On the left panel, we selected Administrator instead of Site from the first dropdown.

  • We searched for Toolbar in the search box, and then we clicked on the red X next to Toolbar to enable the module.

  • We then tried to edit a random article to check that the Save button was there (along with the other buttons in the toolbar) and it was. The problem was solved!

Of course, the million dollar question right now is what caused it to be disabled in the first place? Our answer is, we don’t know, and the client wasn’t really interested in knowing so we couldn’t do any investigation to discover how it was done. But we suspect it was done by someone who had legitimate access to the website, either intentionally or unintentionally. But again, we can’t be sure.

If you can’t see the Save button in the backend of your Joomla website, then most likely your Action Toolbar was disabled. You can re-enable it by following the above guide. If it doesn’t work for you, then make sure you delete your Joomla administrator cache (you can do this by wiping out the contents of the administrator/cache folder). If it still doesn’t work for you, then we suggest you contact us. Our fees are affordable, our work is top-notch, and we are reachable 24/7/365!

A ModSecurity Rule to Block Brute Force Attacks on a Joomla Website

At itoctopus, we dedicate a lot of time to research, especially on enhancing Joomla’s security and performance. Today’s post is a fruit of that research, as we have managed to devise a concise ModSecurity rule to block brute force attacks on a Joomla website.

In case you don’t know what ModSecurity is, it is a firewall module that integrates with the Apache web server (among other web servers). What makes it very powerful is its versatility: it has a rules engine which allows system administrators and developers to develop rules to block an IP based on the its activity, and it has some built-in generic rules that can be whitelisted if they cause problems to the hosted application.

Going back to Joomla, most Joomla websites suffer from brute force attacks on the login page of their backend, and, because of that, their administrators resort to some 3rd party extensions that can handle brute force attacks at the application level. The problem with such extensions, however, is the fact that they are extensions, which means the Joomla framework must be loaded everytime the attack takes place.

ModSecurity, on the other hand, blocks brute force attacks at the server level, which means that the offending IP will not even reach the Joomla application, which is much better from server load perspective.

So, how do you block brute force attacks on a Joomla website with ModSecurity?

We’re glad you have finally asked. Here is a step-by-step super-easy to implement instructions on how to do that:

  • ssh to the server as root.
  • Edit the custom.conf file located under the /usr/local/apache/conf/modsec2/ folder (note: this folder location is the one used in a WHM based server, if you’re using something different, then the folder location might be different). You can use vi to edit the file by just issuing this command:

    vi /usr/local/apache/conf/modsec2/custom.conf

  • Add the following code to the custom.conf file:

    <Location /administrator>
        SecDefaultAction phase:2,deny,status:403,log,auditlog
        SecRule IP:bf_counter "@eq 5" "id:1000002,phase:2,log,block,expirevar:IP.bf_counter=3600,msg:'IP address blocked because of a suspected brute force attack on the Joomla website'"
        SecRule ARGS:option "@streq com_login" "id:1000000,phase:2,chain,t:none,log,pass,msg:'Multiple Joomla authentication failures from IP address', setvar:IP.bf_counter=+1"
    </Location>

  • Save the file and then restart Apache using the following command:

    service httpd restart

  • Try to login to the Joomla website 5 times with wrong credentials and, on the sixth time, you will be directed to a 403 Forbidden page.

  • Now go back to sleep knowing that malicious users will not exhaust your server with their incessant brute force attacks!

Now, of course, you might be wondering what that gibberish code that you added to the custom.conf file really means, so let us explain…

This line…

SecDefaultAction phase:2,deny,status:403,log,auditlog

…specifies what action will be taken when the rule blocks the IP. In this line, we are telling ModSecurity to deny access to the page with a 403 error. Not much else you need to know here.

This line…

SecRule IP:bf_counter "@eq 5" "id:1000002,phase:2,log,block,expirevar:IP.bf_counter=3600,msg:'IP address blocked because of a suspected brute force attack on the Joomla website'"

Checks the variable bf_counter if it’s equal to 5. If it is, then the following happens: rule number 1000002 is triggered (you can give the rule any number you want, provided that number is not already assigned to another rule), we log the event (using the log instruction), we block the IP (using the block instruction), we remove the bf_counter variable after 3600 seconds (hence the expirevar:IP.bf_counter=3600 code – which means that the attacker will be blocked from accessing the page for 3600 seconds), and then we specify which message ModSecurity should have in the logs (which is this one “IP address blocked because of suspected brute force attack on the Joomla website”)

This line…

SecRule ARGS:option "@streq com_login" "id:1000000,phase:2,chain,t:none,log,pass,msg:'Multiple Joomla authentication failures from IP address', setvar:IP.bf_counter=+1

…is the heart of the blocking logic and it took us a lot of time just to know what to have in there. You see, when you login unsuccessfully to a Joomla website, com_login will be a post value passed back to the login page. This value is not passed to any page once the login is successful, which means that every time we see com_login in a post value, it means that we have an unsuccessful login, and we must increment the login counter. The line above does that the following way: it checks if any of the ARGS variables (any post or get variable) has com_login in its value (@streq com_login), if it is, then we define rule 1000000, which will increment the bf_counter by 1. The bf_counter variable is checked by the previous rule: once above 5, the request will be blocked.

Can the ModSecurity rule above be modified to be more lenient just so false positives are not blocked immediately for an extended period of time?

Yes! For example, if you want to change the number of retries to 10 and reduce the time that it takes to reset the number of retries to 30 minutes (1800 seconds), then all you need to do is change the second line in the rule (well, technically the third line if you count the <location… line as the first line) to:

SecRule IP:bf_counter "@eq 10" "id:1000002,phase:2,log,block,expirevar:IP.bf_counter=1800,msg:'IP address blocked because of suspected brute force attack on the Joomla website'"

Are there any caveats?

Well, as a matter of fact, there are 2 caveats:

  1. The above rule effectively makes com_login a blacklisted word, which means that if you have an article with com_login anywhere in its body, and you save it, then every save will count towards the bf_block limit. Of course, the absolute majority of Joomla websites will not use the word com_login anywhere in their content, but some technical websites may.
  2. We are not resetting bf_block when the login is successful, which means that if, you try to login 3 times with wrong credentials, and then you login successfully, and then you logout, and then you login 2 times with wrong credentials, then you will be blocked on the 6th attempt.

We hope you enjoyed this post the same way we enjoyed researching it and writing it (note that we used the book ModSecurity Handbook which is written by one of the programmers who heavily worked on ModSecurity). If you need a little help with the implementation, then we can definitely help. Just contact us and we’ll do it for you swiftly, cleanly, and for a very affordable fee!

The Mystery of the Joomla Login Timeout

Back in May (today is the 21st of June, the day that should have been the summer solstice, but instead it was yesterday, since this year is a leap year [you learn something new every day!]) we had a client who approached us with a very weird problem. The client’s problem, to put it simply, was the sudden inability to login to both the backend and the frontend. In short, the login worked, and then, all of a sudden, it didn’t (it just timed out). After a while, it started working again only to stop working shortly after.

From the get-go, we felt that this had something to do with an aggressive firewall, as firewall issues are usually characterized by erratic problems. What confirmed our theory was the fact that the problems started happening not long after the website was temporarily moved to a different server.

Since we didn’t have root server access to the temporary hosting (and neither did the client, for that matter), we asked the client to authorize us to move her website to another high-end host that we trust, and she did. So, we moved the website, we tested the website (including the login), and everything worked smoothly! We were happy and the client was happy, for a while…

The next morning the client called us and told us that she was experiencing the exact same issue: her clients were not able to login. That, of course, meant that our firewall theory was wrong, and there was something else causing this problem. So, we checked what was going on, and we noticed that there was an authentication plugin connecting to Paradiso which is an online LMS. The way how that plugin worked was a bit complex: it authenticated the login on Paradiso, and then Paradiso authenticated back the login on the joomdle extension on the Joomla website. Disabling the plugin fixed the problem, but, as you might have probably guessed, it also broke the most important functionality that the website had, which was the LMS integration (needless to say, this made the website useless).

Investigating the issue further proved that the login was timing out when the Paradiso authentication plugin (on the Joomla website) was connecting to the Paradiso website (in order to post the login credentials), so we thought that the issue was definitely on Paradiso‘s end. So we asked Paradiso to check the logs on their end to see what’s going on, but they told us that everything was normal on their end, and that the client’s server IP was not blocked.

So, we just took the Paradiso URL and tried to load it directly (from the browser), and it worked. Hmmm… We were still not convinced and we still thought it was Paradiso‘s problem. Nevertheless, we decided to check our logs, and, to our surprise, we found the following:

2016-05-19 15:02:22 SMTP connection from [52.36.77.46]:38053 (TCP/IP connection count = 1)
2016-05-19 15:02:32 SMTP call from us-west-2.compute.amazonaws.com [52.36.77.46]:38053 dropped: too many syntax or protocol errors (last command was "?4?2?\016?\r?\031?\v?\f?\030?  ?")
2016-05-19 15:22:53 SMTP connection from [52.36.77.46]:38397 (TCP/IP connection count = 2)
2016-05-19 15:23:03 SMTP call from us-west-2.compute.amazonaws.com [52.36.77.46]:38397 dropped: too many syntax or protocol errors (last command was "?4?2?\016?\r?\031?\v?\f?\030?  ?")
2016-05-19 15:23:06 1b3TXG-0005Wz-GJ <= root@host.[ourclientdomain].com U=root P=local S=2865 T="lfd on host.[ourclientdomain].com: blocked 52.36.77.46 (US/United States/us-west-2.compute.amazo" for root

Aha! So, it was our client's server that was blocking Paradiso's IP, and not the other way around, and it was blocking their IP because it was sending malformed syntax to the mail server, which explained this whole mess. Finally! After working on this problem for days we found out what the exact cause was!

So, how did we fix the problem?

Fixing the problem was easy - we just whitelisted Paradiso's IP and we then told Paradiso to fix their mail sending program, which they said they will do immediately (we can't confirm whether they fixed it or not).

And why wasn't this a problem for our client before?

Most likely because our client's website resided on a server with no firewall, or, at best, with very loose firewall settings. Any half-decent firewall should block malformed requests to any server application, which incidentally, brings us to this question: why didn't other Paradiso customers complain about this and why was our client the only one affected with this? Obviously, this is a rhetorical question, but a valid question nonetheless.

In any case, we hope that you found this post helpful. If you are facing the same problem on your Joomla website, then check your server logs as the problem may be at the server level. If you can't find anything in the logs or if you need help, then please contact us. We will find what the problem is, we will fix it for you, you won't pay us much, and most likely, you will gain new friends (that's us)!

On Wicked Joomla Developers

It is Friday afternoon, and we have a little present for you: a free real life suspense story about a wicked Joomla developer, how he crashed his client’s website, and how we knew who he was, Sherlock Holmes style. Coincidentally, this post comes after our bad Joomla developers post, which makes sense, as a wicked developer is also a bad developer, but with decisively more intentional destructive capabilities. Are you ready? Let’s start!

A client of ours called us on Monday morning and told us that he’s seeing an error message every time he visits the frontend of his Joomla website. So, we checked his website and we got the following error:

The page isn’t redirecting properly

This meant that there was an infinite loop somewhere causing the page to redirect to itself. So, we checked the .htaccess file for any suspicious redirects, but we found none. Nevertheless, we renamed the .htaccess to .htaccess.old just to make sure that the problem is not related to anything in the .htaccess file. We then visited the website, and, as we expected, we still saw the same error.

We then checked all the plugins installed on the Joomla website to see if there was an SEF plugin or any plugin that is responsible for redirecting, but we found none.

We felt that we were stuck (typically these issues are caused by either a wrong rule in the .htaccess file or an SEF plugin) – and we wanted more information about what’s really going on when someone visits the website. So, we checked the Apache logs under /usr/local/apache/domlogs/[domain-name] and we searched for our IP there (using a simple grep), and we noticed that for each page load there were 20 301 redirects (which is the default Apache limit), until we were redirected using a 303 redirect (temporary redirect) to the Error page. Unfortunately, these findings were not helpful because we knew that that was the case.

We then thought, it must be a weird system plugin causing this redirect (a plugin that we don’t know of). So, we renamed the folder system under the plugins folder to system_old, but that didn’t help: the website still showed the same error message.

We did the same thing with the content folder, maybe there is a redirect happening there, but still, nothing. We even renamed the whole plugins folder to plugins_old, thinking that it will solve the problem (or at least force the website to show an error message complaining that it’s not able to load a specific plugin), but that didn’t do anything: we had the same “The page isn’t redirecting properly” error.

At this point, we started to lean toward the hack theory. So, we logged in to the shell and we generated a list of all the modified files in the past 10 days using the following command:

find /home/[cpanel-user]/public_html -path '/home/[cpanel-user]/public_html/cache/*' -prune -o -mtime -10 -ls | grep -vE '(/cache/)'

The above just listed some images that were uploaded in the past 10 days by the website’s staff – which meant that there were no core files that were changed. However, the above shell command may not always yield accurate results, because a malicious script (once uploaded) can change the date of any file using the touch shell command. For example, to change the modification date of a file called file1.php to 90 days ago, all you need to do is to issue the following command:

touch -d "90 days ago" file1.php

So, we decided to overwrite all the Joomla core files (just in case) using our famous, super quick, super efficient method to cleanup Joomla websites. But, there was a problem, the client had many core files modified (for many reasons) and he didn’t have a list of which files were modified. So that option was out of the question…

Now our best option was to discover the hack the old, manual way: we renamed every folder under the Joomla website from foldername to foldername_old, and then tested the website after each folder renaming to see how it responds. If it crashes the same way, then this folder has nothing to do with the problem, if it crashes by complaining that there is a missing file, then this means that files in this folder are being used before (or during) the hack (in that case we have to rename back that folder to its original name).

So we renamed the folders administrator, cache, components, cli, images, language, layouts, media, modules, plugins, templates, tmp to administrator_old, cache_old, components_old, cli_old, images_old, language_old, layouts_old, media_old, modules_old, plugins_old, templates_old, tmp_old respectively. The folders includes and libraries caused the Joomla application to complain of missing files when renamed, so we had to revert back their names to what they where.

Now, it was obvious that the problem was either in the includes or in the libraries folders. We quickly checked the includes folder (this folder only has a few files), and it had nothing malicious. So now we’re left only with checking the libraries folder for anything malicious.

After a lot of hard and complicated work, we finally nailed it. It was the document.php file (located under the libraries/joomla/document folder) which had the following code in its very beginning:

defined('JPATH_PLATFORM') or die;
$ExpireDate=strtotime("10-06-2016");
$Today=strtotime(date("d-m-Y"));
if($Today > $ExpireDate)
{
	$app=JFactory::getApplication();
	$app->redirect('index.php','');
}

The above code caused the website to redirect to the homepage (causing an infinite loop) if the current date was past June 10th, 2016 (it was June 13th when we worked on this problem).

Now, we’ve seen too many Joomla hacks in our life, but this one was curious: it didn’t seem like a blind hack, it didn’t even seem like a hack at all (that’s why we published the code, because technically, it wasn’t malicious, even though realistically, it caused the client’s website to crash). It seemed as if almost one of the developers working on this website intentionally added the above code (for a mysterious reason). Maybe it was a bug – but why would a high level Joomla developer work on such a critical file?

We informed the client of our findings (of course, we removed the code from the document.php file, thus fixing the problem), and the client instructed us to investigate the issue further (which was great since we are rarely instructed to do forensics). So, we checked the Apache logs around the time the file was modified (to see whether the file was modified by an external script), but we found nothing suspicious (note that the file was last modified on May 29th, 2016 at 20:56 server time [which was EST]).

We then checked the messages file under /var/log for any FTP activity that happened during the time where the file was modified, and we found this:

May 29 20:55:28 c124c pure-ftpd: (?@[ip]) [INFO] New connection from [ip]
May 29 20:55:30 c124c pure-ftpd: (?@[ip]) [INFO] [user@domain] is now logged in
May 29 20:55:34 c124c pure-ftpd: ([user@domain]@[ip]) [NOTICE] /home/[cpanel-user]/public_html//libraries/joomla/document/document.php downloaded  (21324 bytes, 31.55KB/sec)
May 29 20:56:09 c124c pure-ftpd: ([user@domain]@[ip]) [NOTICE] /home/[cpanel-user]/public_html//libraries/joomla/document/document.php uploaded  (20250 bytes, 4.60KB/sec)

Obviously, the file modification happened using FTP by a developer working for our client. Now, of course, one might think that the developer’s FTP credentials were stolen, but we were able to confirm with our client that it wasn’t the case because the IP matched the developer’s location (and, more importantly, it matched the sender IP in the emails sent by the developer to our client).

But, what if the developer was trying to do something and the whole thing was a bug? Well, the fact that it took the developer 35 seconds to add 5 lines of code meant that he copied and pasted the code from somewhere else and the whole thing was intentional. Additionally, the logs (in the messages file) showed that the developer was working on a 3rd party extension that had nothing to do with the core document.php file.

We have to say that the client was both surprised and devastated, as he relied on that developer on many things and trusted him with his own business. The client, despite the huge stress, took the right decision by not confronting the developer with our findings (thus avoiding a potential backlash), and he smartly allocated the developer on another (low-priority) project he has, just after having us lock down all access to the website.

Unlike the client, we were not surprised because we know that there are some developers out there who are wicked, who lack basic professional ethics, and who are typically physically located outside their client’s legal jurisdiction (most of these developers are located overseas), so they can do whatever they want with zero consequences (at least for them) and for any reason (in this particular scenario, the client explained that it was possible that the developer wanted to break the website so that he can get paid for fixing it), but these developers are seemingly cheap, and there’s always a risk when it comes to cheap labor: you might be lucky and get a super developer who will do whatever you want for pennies, or you might be unlucky and get this sleazy, sneaky, wicked developer who will end up costing you much more money than a real developer.

At itoctopus, our clients love us because we are ethical Joomla developers. We are fair in our estimates, we try to save money for our clients, our code is clean and well tested, and we never have our own interests in mind when providing advice to our clients. So if you need real, honest, affordable, and professional Joomla developers, then please contact us, and we promise you that it’ll be the start of a long, rewarding, and positive business relationship!

Let’s Not Forget About Bad Joomla Programmers

Most people using Joomla are very quick to blame it (Joomla) for any hiccups/problems they see on their websites. In most cases, these people are wrong. In fact, the majority of the problems happening on Joomla websites are not technically caused by Joomla’s core, but they are caused by one of the following:

  • A change in the server environment
  • A missing library on the server or inadequate server settings
  • Ignoring security-critical Joomla updates
  • Bad Joomla programmers

In this post, we would like to focus on bad Joomla programmers, and how they negatively affect the performance of a Joomla website and the perception of Joomla by the general public. Let us start by giving you an example…

Early last week, a client of ours told us that he was seeing the Allowed memory size error on many of his website’s pages, and he emailed us an example of a link so that we can see the problem. We clicked on the link and we saw the following error:

Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 130968 bytes) in /home/[user]/public_html/libraries/joomla/database/driver/mysqli.php on line 515

Typically, fixing the Allowed memory size problem consists of just increasing the PHP memory limit in the main index.php file, or in the .htaccess file, or in the php.ini (o .user.ini file). But, no matter how much we increased the memory limit for that particular case, we still experienced the same fatal error.

We then looked closer at the error, and we noticed that it was complaining that it was running out of memory in the mysqli.php file, which meant that the error had to do with MySQL. So we looked at the MySQL slow query log, and we noticed nothing unusual, but we were still sure that there was something weird happening on MySQL, and so we enabled full logging (e.g. all the queries hiting the MySQL database server would be logged) the following way:

  • We ssh’d to the server as root.
  • We added the following code to the my.cnf file (which is located under the /etc folder):

    log = "/var/lib/mysql/all-queries.log"

  • We opened the .htaccess file located under the main directory of the website, and then we denied access to all traffic with the exception of our IP (it is not practical to allow full logging from all IPs).

  • We restarted MySQL by issuing the following shell command:

    service httpd restart

After enabling the logging of all queries, we visited the problematic link, and, in literally 5 seconds, we had 18982921 bytes in the all-queries.log file, and it was full of the following query:

SELECT parent_id,title FROM #__categories WHERE id =0

In fact, the above query was repeated 270,000 times (yes, you read that right, two hundred seventy thousand times, a bit more than a quarter of a million), and that was just after loading one page! Wow!

A quick search for the above query revealed that it was in a very poorly written custom extension (e.g. the extension was specifically written for that Joomla website), and more specifically, the query was in a recursive function which job was to find the title of the ultimate parent of a category. The problem with that function is that it lacked the most basic characteristic of a recursive function: it had no stop condition!

As programmers, we know how to write recursive functions with ease (a recursive function is a function that calls itself until a stop condition becomes true – if you are interested in programming and you want to learn more about recursive functions, then the best way to learn it is by reading the chapter on recursion in David S Touretzky‘s COMMON LISP: A Gentle Introduction to Symbolic Computation), but we know that recursive functions are not for everyone. In fact, what distinguishes a real programmer from a quack one is the ability to grasp the concept of recursive functions. Obviously, whoever wrote that function was a quack developer, because his function missed the most basic concept of recursivity.

Fixing the problem simply consisted of just adding the correct stop condition to the recursive function. Ironically, after fixing the problem and informing the client of the root cause, the programmer who developed the custom extension emailed us and thanked us for fixing his code (by the way, the client told us in his first email concerning this issue that this same programmer searched for weeks for the cause of the problem but he couldn’t find it), promising the client (who was cc’d on the email) that he will pay more attention in the future. This meant that the client wanted to continue the business relationship with that programmer, despite the obvious fact that he wasn’t a real programmer. The only logical explanation to this is that this programmer was cheap, and that is the very heart of the problem affecting so many Joomla websites (especially the large ones) out there: cheap, unreliable, and unprofessional Joomla programmers.

So how can Joomla get rid of this problem?

The short answer is that it can’t: there is no way Joomla can prevent people from dealing with cheap programmers. In fact, this problem is not specific to Joomla, it is a problem that applies to any work under the IT umbrella, except that there is a general perception and skepticism in the Joomla community and userbase blaming Joomla for everything (well, not really everything, the hole in the ozone layer is blamed on hair spray and air conditioners).

The long answer is that Joomla can minimize the problem by certifying Joomla developers (this is substantially different from certifying Joomla administrators). We’re not talking about writing a simple plugin here; we’re talking about having an actual computer science university degree and a vast LAMP experience. Of course, the technicalities are sketchy at best at first glance, but, with enough resources (and maybe with forging some alliances with PHP and MySQL), Joomla can get this certification program up and running. Now, of course, this will not solve the problem, because many Joomla administrators out there want cheap labor but still expect expert work (it’s like driving a Mazda but expecting a Mercedes experience), but it is a start!

Meanwhile, if you’re looking for expert Joomla developers to fix/implement anything on your Joomla website, then look no further. We’re here to help! Just contact us and let us demonstrate to you that we’re fast, that we’re reliable, and that although we’re not cheap, we’re quite affordable!

A Quick Method to Disable 2 Factor Authentication on Joomla’s Backend

It’s been a long while since you have last updated the content of your Joomla website, but you decided, on a Sunday afternoon, to do just that.

So, you navigate to the login page of the backend of your Joomla website, and you see a curious “Secret Field” input field. You ignore it, and you just try to login with your good old username and password, but when you do that, you see the following error message:

Warning: The two factor authentication Secret Key is invalid.

You try to login again, but you see the same error, and you finally realize that you just can’t leave that “Secret Field” input field empty, but you don’t know what to put in there. So, you start searching your emails and any documents that you have for that “Secret Field”, but you find none: someone who had (and may still have) access to the backend of your Joomla website added Two Factor Authentication to the login but didn’t tell you anything about it (maybe he forgot to do that or maybe he didn’t).

Since you’re not sure who that “someone” is (or maybe you’re sure but that “someone” has gone MIA), you decide to disable that Two Factor Authentication, but you don’t know how. Luckily, we’re here to help: here’s a concise and comprehensive (no, that is not an oxymoron, we checked) guide on how to do that:

  • Login to phpMyAdmin through your WHM/cPanel/other account.
  • Select the database powering the Joomla website.

  • Run the following queries:

    UPDATE `#__extensions` SET `enabled` = '0' WHERE `name` = 'plg_twofactorauth_totp';
    UPDATE `#__extensions` SET `enabled` = '0' WHERE `name` = 'plg_twofactorauth_yubikey';

    Note: You should replace #__ with your table prefix.

  • That’s it! Close your browser and then go to the login page again and you will see that the “Secret Field” is gone.

Is there an easier way to disable the Two Factor Authentication?

Well, no, unless, of course, you can access the backend of your Joomla website with an empty “Secret Field” (or you already know what the “Secret Field” is). In that case, you can just disable it the following way:

  • Login to the Joomla backend.
  • Click on Extensions -> Plugins.

  • Under Select Type on the right side select twofactorauth.

  • Disable all the plugins that are enabled there, namely Two Factor Authentication – Google Authenticator and Two Factor Authentication – YubiKey (you can easily disable a plugin by clicking on the green checkbox next to its name).

  • That’s it!

We hope that you found our guide on disabling the Two Factor Authentication useful. If you need help with the implementation, then please contact us. We will do it for you swiftly, professionally, and affordably. You will also gain true and reliable friends in the Joomla realm!

How to Resolve the “Call to undefined function mb_split()” Fatal Error on a Joomla Website

We were commissioned yesterday morning to move a website form GoDaddy’s shared hosting to a VPS (with another company). So, we zipped the files and we exported the database from the old server, and then we extracted the files and imported the database on the new server.

We then proceeded with testing the website (after modifying the configuration settings to make them point to the database on the new server), but, the moment we loaded the website, we saw the following error:

Fatal error: Call to undefined function mb_split() in /home/[user]/public_html/components/com_roksprocket/lib/RokSprocket/Util/HTMLManipulator.php on line 36

Since we have encountered this fatal error many times before, we knew exactly what’s causing it and we knew exactly how to fix it.

So, what’s causing it?

Well, typically companies leasing VPS’s (or dedicated servers) to clients provide these VPS’s with the basic settings – so these VPS’s don’t have many Apache/PHP modules installed (they just have the very basic ones installed). Even common PHP modules are not installed, such as the Mbregex module (which stands for Multibyte Regular Expression, which is used to manipulate strings in all languages, not just Western languages), which absence is causing the fatal error.

OK, so how you can fix it?

Fixing the problem consists of merely installing the Mbregex PHP module on the VPS. Here’s a quick guide on how to do this (we are assuming you are running a WHM based VPS):

  • Go to WHM.
  • On the left side, search for EasyApache3, and then click on the EasyApache3 link.

  • Click on the wheel next to the selected profile to modify the settings.

  • Click on “Next” until you reach the “Short Options List” tab.

  • Click on “Exhaustive Options List” button (do not click on the “Exhaustive Options List” tab – the button that you should click is located at the very bottom of the page)

  • Search for Mbregex in the list and click on the checkbox next to it.

  • Click on “Save and Build” at the bottom of the page.

  • Click on “OK” on the popup window, and then click on “I understand” on the second popup (the second popup is telling you that you should not terminate the process or you may risk corrupting the Apache server installed on your host).

  • Wait for about 15 minutes and don’t do anything (it is recommended that you don’t even use your machine for anything until the process is completed).

  • Click on the “Save New Configuration” button on the final popup window (note that you must scroll down in the popup to locate this button).

  • That’s it! Your problem should be solved (we know it solved it for our client and for many other clients).

If you’re facing the same fatal error on your Joomla website after migrating to a VPS (or a dedicated server), then follow this guide and you should be all set. If this is happening to you after moving to a shared host, then you should contact your host (almost all hosts are accommodating when it comes to missing libraries, which is a good thing and a bad thing). If you think this guide is a bit too technical for your taste, or if you are running something different than WHM (such as Plesk), then please contact us. We will fix the problem for you in no time and for a super affordable fee!

GoDaddy Shared Hosting and Hacked Joomla Websites

We are having an increasing number of calls from clients asking us to fix their hacked GoDaddy shared hosted Joomla website. While many may see this as an indication that GoDaddy has many customers, we see the whole thing a bit differently.

The thing is, a few months ago, we only had a few GoDaddy hosted hacked Joomla websites to fix every month, but these days, we’re getting 3-4 a day! So, it’s not like GoDaddy’s clients jumped 30 times overnight, and it’s not like our clients jumped 30 times overnight. Something’s fishy, and, so we decided to investigate further.

It didn’t take us long to get down to the bottom of the problem, GoDaddy’s shared hosting servers all (well, at least all the ones that we have worked on) suffered from the ImageMagick vulnerability. GoDaddy, so far, has not taken the appropriate action and patched the ImageMagick exploit.

Not only that, our clients tell us that GoDaddy, when they are informed of their clients’ misfortune, try to talk them into buying a “security product” to make their websites more “secure”, essentially blaming the hack on the absence of security products installed on the website.

While we have the utmost respect for GoDaddy, we don’t think they’re doing the right thing here, one could see them as extorting money from their clients because of a server exploit that these clients cannot control.

So, what do we do when get a GoDaddy hosted hacked Joomla website?

We do the exact same thing that we do for other other hacked websites: we cleanup and secure the websites, except that at the end, we tell the client that they need to move their websites to a VPS (they can remain with GoDaddy, but they just need to move to a VPS).

We think that GoDaddy is making a mistake with their current perceived strategy of “make-the-client-pay-for-our-server-exploit”, but they are a gigantic company, and maybe they know something that we don’t, or maybe the whole thing is not intended, or maybe we are just plain wrong (and we hope so).

Whatever it is, we’re sure that GoDaddy clients are not finding it fun to get hacked. So, if you are a GoDaddy customer with a hacked Joomla website, please contact us and we’ll clean your website for you in no time and for a very affordable cost!

Resolving the Dreaded “Call to a member function getDocument() on a non-object” Error on a Joomla Website

A client called us yesterday and told us that his website was displaying the following error after a manual upgrade of his Joomla 3.1.1 website to Joomla 3.5.1:

Fatal error: Call to a member function getDocument() on a non-object in plugins/system/sef/sef.php on line 36

We then asked him, what do you mean exactly by a manual update? He said that he just downloaded Joomla 3.5.1 and then extracted it to the main directory of his Joomla 3.1.1 website. Hmmm… Typically, such an update can cause problems because Joomla’s 3.5.1 file structure is slightly different (actually “slightly” is not the right word, it’s slightly more than slightly) than that of Joomla 3.1.1.

Naturally, the first thing that we did (after asking the client if he had a backup of the full website before updating, which unfortunately he didn’t have) was that we opened the sef.php located under the plugins/system/sef folder, and we looked for the error line (39), which was this one:

$doc = $this->app->getDocument();

We did some printouts, and we discovered that $this contained a lot of data, but not $app, which was curious, since $app was a protected attribute, which meant that the parent class, JPlugin, must have it defined somewhere.

So we opened the file plugin.php, which was located under the libraries/cms/plugin folder, and we checked what was going there. We saw that it was defining $app under certain conditions, in the following code:

if (property_exists($this, 'app'))
{
	$reflection = new ReflectionClass($this);
	$appProperty = $reflection->getProperty('app');

	if ($appProperty->isPrivate() === false && is_null($this->app))
	{
		$this->app = JFactory::getApplication();
	}
}

So, in order to see if the above conditions were met, we added a die(‘in condition…’); just after the first condition, but that didn’t print out. So we added die(‘in file…’); at the beginning of the plugin.php file, but that didn’t print out either, we still saw the same issue, which essentially meant that the plugin.php file wasn’t even loaded at all! Curious and curiouser… How can a child class run without loading the parent class first? That’s practically impossible!

The next logical step was to confirm that the aforementioned plugin.php file was not loaded by Joomla (to make sure that we are not running into insane caching issues). In order to do that, we added the following code in the sef.php file immediately before the problematic line:

$arr = get_included_files();
print_r($arr);
die();

We then checked the website, which had a list of all the files that were loaded by the Joomla application to serve that page. We particularly searched for the file plugin.php, and we found it being loaded from this location:

libraries/joomla/plugin/

But, the file should be loaded from here:

libraries/cms/plugin

A quick check in the Joomla 3.5.1 default directory revealed that the folder libraries/joomla/plugin doesn’t even exist anymore, and that that folder was ported over from the previous Joomla installation. So, the problem was that Joomla was loading the JPlugin file from the old directory, and not from the new directory.

So, how did we fix the problem?

Fixing the problem consisted simply of deleting the folder libraries/joomla/plugin. But, we had other problems after fixing this problem, which we resolved by removing the following folders:

  • libraries/joomla/html
  • libraries/joomla/pagination
  • libraries/joomla/registry

We then had to repair the database (twice).

As you can see, fixing a botched Joomla update is not that hard (once you know how), but again, it’s always a good idea to update Joomla using the standard, recommended method (as other methods are pretty much advanced because they entail fixing fatal errors).

If you are seeing the much dreaded Call to a member function getDocument() on a non-object then most likely you are left with a folder (or a few folders) from a previous Joomla install. If you need help finding these folders, then please contact us. We will find these folders, delete them, make sure that your website works again, and we will do that for a very affordable fee!

“Error displaying the error page: Application Instantiation Error” Fatal Error on Joomla

Late this evening, a client called us and told us that, after moving his website to a new host, he was seeing the following (fatal) everywhere on his website:

Error displaying the error page: Application Instantiation Error

Typically, the above error is associated with wrong database credentials in the configuration.php file, so we tried the following: we reset the database user password in cPanel, and used the new password in the configuration.php file (after making sure that the host, the database name, and the database user were all correct), and then tested the website again, but still, we saw the same error.

We were almost confident that the problem had to do with accessing the database, and so we did the following:

  • We opened the file mysqli.php (the client was using mysqli as a database driver) located in the libraries/joomla/database/driver/ folder.
  • We removed the @ sign from the following line:

    $this->connection = @mysqli_connect(
    	$this->options['host'], $this->options['user'], $this->options['password'], null, $this->options['port'], $this->options['socket']
    );

    (Note: the @ sign suppresses any non fatal error from being displayed, which means that removing it will display the actual, real error.)

  • We loaded the website again, and this time we saw the following error:

    Warning: mysqli_connect(): (HY000/1045): Access denied for user ‘[user]’@’localhost’ (using password: YES) in libraries/joomla/database/driver/mysqli.php on line 141

The Access denied in the above error revealed what the actual issue was: the user was not given the proper permissions to the database. So, we went back to cPanel, we clicked on MySQL databases, and then, under Add User To Database, we added the appropriate user to the Joomla database.

If you’re seeing the same error on your Joomla website, then it’s almost a certainty that it’s a database connection problem. Make sure that all your credentials are correct and also ensure that the database user has full access to the Joomla database. If the problem persists, then please contact us. We will fix it for you quickly, professionally, and affordably.

Joomla Backend Login Redirects to “Sorry” Page on WHM/cPanel

Yesterday, we were commissioned to move a large website (for a US clinic) from one server to another: we moved the files, we installed the database, we modified the configuration.php file to reflect the new server’s settings, we modified our Windows hosts file (to force the domain to point to the new IP), and then we started testing the website.

Everything worked very well, but, when trying to login to the backend of the Joomla website, we were greeted with the default WHM/cPanel Sorry 404 page. We quickly noticed that the website was being redirected from http://www.[ourclientjoomlawebsite].com to https://www.[ourclientjoomlawebsite].com/, which meant that the problem was with the secure mode of the website.

A very quick research revealed that this problem occurs when the SSL certificate is not installed on the server, which was exactly our case.

So we had 2 options: We either had to install the SSL certificate on the new server or we had to make the admin backend work in normal (non-https) mode. We chose the latter because we didn’t have the SSL certificate. Here’s what we did:

  • We logged in to phpMyAdmin.
  • We clicked on the #__modules table.

  • We searched for the module named Login that has the field client_id set to 1 (typically the id of this module is 2).

  • We changed the following string in the params field from:

    "usesecure":"1"

    to

    "usesecure":"0"

  • We saved the row (module) and then we closed the browser (note: you may need to also clear the browser cache in FireFox), and then we launched the browser and tested the login.

  • The login worked!

As you can see, solving the problem was quite easy. The alternative solution, of course, is to install the SSL certificate. But, if you don’t have to the SSL certificate, then the quick solution is the one we explained in this post. If you need help implementing this solution (or the other solution), then please contact us. We work 24/7/365, we work professionally, we work quickly, and our fees are super affordable!

How to Use Joomla’s Ajax Interface Component (com_ajax)

Last week, we were commissioned to build a tool that displays a list of state representatives on the website of a company producing soda dispensers. The tool consisted of a simple dropdown, containing all the American states, and, based on the selected state from the dropdown, the tool displayed the representatives serving the selected state.

Of course, there are many ways to do this, but we decided to do this using Ajax, but not just any Ajax; we wanted to use Joomla’s Ajax interface (yes, we know that Ajax is the same across the board, but you know what we mean).

Now, before telling you how we did it, let us explain a bit what Joomla’s Ajax interface is… You see, Ajax, at its purest and simplest form, consists of a request to a server and a response from a server, the request is made through JavaScript, and the response is sent through JavaScript, but typically, the latter is generated on a server by a PHP file (in the case of a PHP based environment). In order to avoid clutter and to stick with the MVC pattern, Joomla comes bundled with a component called Ajax Interface, which allows the programmer to get the response data from a module or a plugin. Unfortunately, many Joomla programmers out there don’t know that this component actually exists, so they create non-Joomla PHP files on the server in order to serve Ajax responses. What’s even more unfortunate, is that most of the remaining programmers that know about Joomla’s Ajax interface are intimidated by it (we have to admit we were among these people), mistakenly believing it’s a monster that is hard to work with, while the reality is the absolute opposite (and we are going to demonstrate that in this post).

So, how did we do it?

Well, the first thing that we did was create a K2 module called K2 Representatives (mod_k2_representatives), that queries the #__k2_items database for specific items where an extra field matches the value of the state passed as a GET variable in the PHP $_GET array.

The module consisted of the following 4 files:

  • index.html: An empty HTML file which should be in every Joomla folder for security reasons.
  • mod_k2_representatives.xml: This is the XML manifest of the module – standard stuff.

  • mod_k2_representatives.php: This is the main module file. It just contained the following 3 line:

    defined('_JEXEC') or die ;
    require_once (dirname(__FILE__).'/helper.php');
    $items = modK2RepresentativesHelper::getItems($params);

  • helper.php: The module helper file. This file contained just 2 methods (within the modK2RepresentativesHelper class) which handled all the logic of the module. These methods are:

    • public static function getItems(&$params, $format = ‘html’): We copied this function from the helper file of the K2 content module and we modified it to accommodate our needs. The function returned the items with extra items matching the GET state.
    • function getRepresentativesAjax(): This function called the static getItems function, and transformed the returned data into a well formatted string.

We then created a Joomla article containing the following code:

<script type="text/javascript">
var jQueryRepresentatives = jQuery.noConflict();
jQueryRepresentatives(document.body).on('change','#state',function(){
	jQueryRepresentatives.ajax({
		type: 'GET',
		url: "http://www.[ourclientjoomlawebsite].com/index.php?option=com_ajax&module=k2_representatives&format=raw&Itemid=300&method=getRepresentatives&state="+jQueryRepresentatives('#state').val(),
		success:function(data){
			jQueryRepresentatives('#results').html(data);
		},
		error:function(){
			jQueryRepresentatives('#results').html('<p class="error">An error was encountered while retrieving the representatives from the database.</p>');
		}
	});  
});</script>

<form action="" method="post"><strong>State:</strong>
	<select id="state">
		<option value="NA">Select a State</option>
		<option value="AL">ALABAMA</option>
		<option value="AK">ALASKA</option>
		<option value="AZ">ARIZONA</option>
		...
	</select>
</form>
<div id="results"> </div>

Now – the essence of the above is this line:

url: "http://www.[ourclientjoomlawebsite].com/index.php?option=com_ajax&module=k2_representatives&format=raw&Itemid=300&method=getRepresentatives&state="+jQueryRepresentatives('#state').val()

  • The module GET variable should contain the name (not the title) of the module to get the data from. In our case, it is k2_representatives (notice that it doesn’t contain the mod part, e.g., it is k2_representatives and not mod_k2_representatives). If you provide the name of a module that doesn’t exist (or is disabled), then the Ajax interface will return the following error:

    LogicException: Module mod_k2_representatives is not published, you do not have access to it, or it’s not assigned to the current menu item.

  • The Itemid GET variable should contain the ID of the menu item that this module is assigned to. The Itemid must be correct – or else you will get the same error as when you provide a wrong/disabled module name.

  • The method GET variable should contain the name of the function that the Joomla Ajax interface should call in order to grab the data from. Now, you may notice that we don’t have a function called getRepresentatives, but, in the helper file, we have a function called getRepresentativesAjax. The thing is, the Joomla Ajax interface insists that any function it calls must end with Ajax. That is OK, because it is possibly done this way in order to avoid conflicts and to allow the developers to have a different output for the Ajax interface (than that of the standard module output). The odd thing is that the Ajax method should not contain the word Ajax (well, technically, it can, but in that case, the method in the helper file should be called something like getRepresentativesAjaxAjax [yes – you should have the word Ajax twice in your method]). So if, in your Ajax call, the method name is myFunction, then you should have in your helper file a function called myFunctionAjax, if you don’t have such function, then you will see the following error:

    LogicException: Method myFunctionAjax does not exist.

  • The state GET variable grabs the current selected state for data filtering.

  • The rest of the GET parameters are static and should not be changed. You should always have option=com_ajax and format=raw in your Ajax request.

The above, as you can imagine, worked as a charm, and the client was happy with the clean end result, and we were happy with the clean way it was done.

If you want to use Ajax on your Joomla website, then follow the above guide and you will not be disappointed. If you need help with the implementation, then please contact us. We are always available, we will finish your job quickly, and our fees are very affordable!

“Login denied! Your account has either been blocked or you have not activated it yet.” Error When Logging in to a Joomla Website

This evening, a new client from Florida approached us with what seemed to be a very easy issue: whenever he tried to login to the backend of his Joomla website with his super user (aka super administrator) account, he was seeing the following error:

Login denied! Your account has either been blocked or you have not activated it yet.

We were happy because after weeks of doing a lot of hard and complex Joomla labor, we were rewarded with a very simple task! We thought that fixing this problem simply consisted of doing the following:

  • Logging in to phpMyAdmin.
  • Clicking on the #__users table (where #__ is the table prefix used by the Joomla website).

  • Searching for the user in question and then changing the value of block from 1 to 0.

But, our hopes of an easy job faded quickly when we saw that the #__users table consisted only of 2 users (admin and admin2), and neither of them was blocked.

So we started debugging this issue by trying to find out the conditions under which the Joomla engine would throw the above error. We noticed that the block logic was only performed in the User – Joomla plugin – specifically in the following lines of the onUserLogin function which is located in the joomla.php file, which is in turn located in the plugins/user/joomla folder:

if ($instance->get('block') == 1){
	$this->app->enqueueMessage(JText::_('JERROR_NOLOGIN_BLOCKED'), 'warning');
	return false;
}

So we thought: let’s, at the beginning of the onUserLogin function (after $instance = $this->_getUser($user, $options);), add the following code…

print_r($instance);
die();

…and then try to login. And so we did, but, to our surprise, we still saw the same error when we tried to login. At this point, we started doubting whether we were working on the right website, and so we added a die() at the beginning of the main index.php file (located under the administrator folder), and the website indeed, died, which meant we were working on the right folder. So this only meant one thing, there is another, non-core plugin, throwing the exact error… Our job now was to find that plugin…

But we didn’t: We created a script to search every single file on the Joomla website to find that error message, yet we only found it in the aforementioned joomla.php file. We then looked into the extensions table in the database, and we checked all the authentication plugins, and we noticed that the plg_authentication_joomla plugin was disabled. Aha!

So we changed the enabled field of the plg_authentication_joomla plugin from 0 to 1 and we disabled the plg_authentication_cookie plugin (which was enabled, for some reason), and then we we closed the browser, and tried to login again, and it worked! (Note that due to some aggressive caching you may have to wait a few minutes before trying to login after re-enabling the plugin).

But how come the “print_r($instance);” didn’t work?

The answer is server side caching. In fact, to prove our point, we re-added the print_r statement followed by the die(); statement, and then we tested it after fixing the problem, and still we weren’t able to see that $instance printed. But, after a few minutes, the mysterious cache expired and we were able to see it. (Note: we noticed that that type of caching is the trait of a cheap, mainstream host, and the Joomla website that we were working on resided on such host.)

But what caused this?

We’re not really sure – but we suspect that someone with a super user login to the Joomla website disabled the Authentication – Joomla plugin either intentionally or unintentionally.

So, if you’re trying to login to your own Joomla website and you’re seeing the Login denied! Your account has been blocked… error, then check if the account is not really blocked in phpMyAdmin (as described above). If it’s not blocked, then check if the Authentication – Joomla plugin is enabled in phpMyAdmin. If it isn’t, then you should enable it. If it is, then please contact us. We will fix the problem for you in no time and for a super affordable fee.

ImageMagick and the Onslaught of Joomla Hacks

As discussed in a previous post, we have been experiencing an onslaught of Joomla 3.5.1 hacks for a few weeks now, and although we were able to clean and protect the hacked websites that we were asked to fix, we were not able to get to the root cause of the problem.

We did, however, suspect that the problem lied within the latest version of Joomla (3.5.1 at the time of writing this post), since the absolute majority of the hacked websites were 3.5.1. This morning, however, we discovered that we were wrong (and we were glad that we were wrong), and that the problem had nothing to do with Joomla, and everything to do with ImageMagick (a Linux software mainly used for image conversion).

So, what happened this morning?

Well, very early in the morning today, we got a call from a new client asking us to immediately clean his company’s website, as it was clearly hacked. The Joomla website didn’t have a single 3rd party extension installed (it was even using the Beez3 template) and it was powered by the latest version (3.5.1). For us, this was another solid proof that Joomla 3.5.1 was vulnerable. So, we fixed the website, we secured it, and then, after being granted permission from the client, we ran some forensics to discover how the website was hacked in the first place.

As soon as we started with the forensics, we noticed that the website was on a shared hosting, which was intriguing, since the absolute majority of the websites that we fixed in the past few weeks resided on a shared host. Of course, it was not logical to say that the cause of the problem is shared hosting, but that was enough to change the direction of our investigation. So we emailed a few of our clients for whom we cleaned Joomla 3.5.1 websites in the past few weeks asking them for permission to check their (server) environment. Luckily, they OK’d our request very quickly (within minutes).

It didn’t take us long to discover that ImageMagick was the root cause of the problem – all the environments with hacked Joomla 3.5.1 websites had ImageMagick installed. In case you don’t know, it was revealed in CVE-2016-3714 that the latest version of ImageMagick had a major exploit allowing for remote code execution (this is very bad), which meant that any application residing on a server with ImageMagick installed was extremely vulnerable, and that was the root cause of why all those Joomla websites were hacked.

So, what is the solution to this problem?

Ideally, the best solution is to move your website to a VPS where you have control over which applications are installed (most servers used for shared hosting have many libraries installed to accommodate the various needs of the hosted websites, and that’s why they are usually more vulnerable). If you don’t want that, then you should tell your host to patch their current version of ImageMagick or risk suffering repeated attacks on your website.

How can you know that ImageMagick is installed on your website’s environment?

There are several ways to do that:

  • If PHP is compiled with ImageMagick, then an ImageMagick section will appear in the output of phpinfo().
  • If ImageMagick is installed but not compiled within PHP, then you can check if you have it installed by running the following PHP script:

    exec("/usr/bin/convert -version", $output, $version);
    echo($version);

    If you get a number (or a combination of numbers), then you know that you have it installed on your server.

  • If you are on a VPS or a dedicated server, then you can try running the following in the shell prompt as root:

    convert -version

    or

    /usr/bin/convert -version

    If you get a valid output (and not something like command not found) when running any of the commands above, then this is a definitive sign that you have it installed.

If your Joomla 3.5.1 is getting repeatedly hacked, then most likely your server is running ImageMagick. Your best option is to cleanup your website and immediately move to a VPS where ImageMagick is not installed. If you need help doing this, the please contact us. We will clean your website for you, we will secure it, we will move it, and we won’t charge you much!

What’s with All Those Hacked Joomla 3.5.1 Websites?

As of late, we are getting an abnormal number of requests for malware cleanup on Joomla websites (a substantial number of these websites are infected with the Google hack). What’s even more interesting is not the sheer number of such requests, but the fact that the absolute majority of the affected Joomla websites were 3.5.1 (which is the latest Joomla version at the time of writing this article).

Of course, this can be a coincidence, but it’s more like a once-in-a-blue-moon coincidence, because 1) the number of hacked websites that we are getting is abnormally large, 2) the absolute majority of these websites are Joomla 3.5.1 websites (typically, hacked Joomla websites use an old/vulnerable version of Joomla), and 3) we all know that Papa Smurf has better things to do than summoning that blue moon all the time.

We hate to say it, but it seems like there is a critical vulnerability in Joomla 3.5.1 that probably only those with malicious intents know about. In fact, we think that this vulnerability, whatever it is, started with Joomla 3.5.0, but since Joomla 3.5.1 was released shortly after 3.5.0, many have immediately upgraded to 3.5.1, which made it seem that this whole mess started with Joomla 3.5.1.

But how come we haven’t investigated the issue?

So far, all of our clients have elected to solely cleanup their websites and secure them. They didn’t want to spend more money for forensics, which is completely understandable, since forensics is a lengthy and a costly process. Of course, we can do the forensics ourselves (without being commissioned by a client to do it), but the problem is that, between all these hacked Joomla 3.5.1 websites, we don’t have the slightest amount of free time to invest in this investigation.

Is it a better idea to stick with Joomla 3.4.8?

We think so. Joomla 3.4.8 is, in our opinion, the most stable Joomla release in the 3.x line (so far). All the Joomla websites that we fully manage are still using Joomla 3.4.8, and we are not intending on updating them any time soon.

How do we ensure that the websites remain clean after unhacking them?

If you have ever cleaned a hacked Joomla website before, you will probably know that the trick is not cleaning it (although that can be very challenging), but it is keeping it clean.

In order to keep those websites clean after removing all the malware, we ensure that the “index.php” file is the only PHP file that can be directly executed by Apache and we remove direct access to most of the core directories (that shouldn’t be accessed by the outside world) by adding a local .htaccess file with a deny from all directive. We also install a homemade firewall extension on the cleaned websites.

Why are we complaining about this? Doesn’t all of this mean that “business is good” for us?

Well, on the long term, this is not good news. If Joomla is deemed to be an unreliable and insecure CMS by the people using it, then these people will slowly, but surely, walk away to other content management systems, and we will be left with less clients. We don’t want that, and we will try our best to get to the bottom of this issue as soon as we have the time. Meanwhile, if your Joomla 3.5.1 website gets hacked, please contact us. We will clean it, we will secure it, and we won’t charge you much!

“The table ‘#__session’ is full” Error in Joomla

A client emailed us this Saturday morning and told us that he was seeing a blank page on every page on his website. Clearly that was the sign of a fatal error. Naturally, all we had to do was to search through the logs in order to find out what the problem is; it didn’t take us long to get to the bottom of it: the MySQL error log (located under the /var/lib/mysql folder) was inundated with the following error:

The table ‘#__session’ is full

Apparently, for some reason, the session table was reaching the maximum limit. That was weird. Typically, on dedicated servers (our client was hosting his website on a dedicated server), the limit of a MySQL table is only restricted by the amount of free hard disk space on the server. So we logged in to phpMyAdmin, and the first thing that we noticed was that the #__session table was using the MEMORY storage engine as described here, and that explained why the table was reaching a limit.

You see, MEMORY tables cannot exceed, in size, the value of the max heap table size global variable in MySQL, which defaults to 16 MB on a standard MySQL installation (if you wan to know the max heap table size on your specific MySQL instance, then do the following: 1) login to phpMyAdmin, 2) click on Variables on the top [without selecting any database], 3) in the Filters search box, enter the word “heap” and then press enter, 4) you will now be able to see what the value of the max heap table size is).

As you might have probably guessed, fixing this problem consisted of just increasing the value of the max heap table size, which we did the following way:

  • We ssh’d to the server.
  • We opened the file my.cnf located under the /etc folder (we always use vi for editing files in Linux).

  • We added the following line to it:

    max_heap_table_size=256M

    (Note: You may already have the above line in your my.cnf file. In that case, all you need to do would be to increase the limit to a higher number.)

  • We restarted MySQL using the following command:

    /etc/init.d/mysql restart

    (Note: You can always restart MySQL from WHM in case you use it).

  • That fixed the problem.

Of course, we truncated the #__session table first, and we recommend that you truncate it in a cron job that runs every midnight, or else you may run into the same issue even after making the change above.

If your #__session table is getting full, and you are using the MEMORY storage engine, then try our solution above, it should work. If it doesn’t, or if you need help implementing it, then please contact us. Our rates are super affordable, our work is super professional, and we are super friendly!

Joomla 404 Pages Causing Load and Memory Issues – How to Fix

Note: The solution presented in this post consists of a core modification, since it addresses a Joomla bug. Proceed at your own risk and always backup your Joomla website before modifying core files. Oh, and don’t forget that future Joomla updates may wipe out your changes (so you may need to redo them after each update).

Another note: Just after finishing this post, we discovered that we have written something very similar here. Oh well!

A problem that is inherent to Joomla, and that most people do not know about, is the 404 page, and it’s not because most Joomla websites do not handle it properly, it’s because the load and memory issues it can cause on large websites. “Load issues”, we hear you ask? What possibly can a 404 page have anything to do with the load on the server? It should be the quickest page to load on the whole website? Right?

Well, wrong! You see there is a bug in Joomla that makes the 404 page the heaviest page on large websites. The problem is that Joomla processes the 404 page as a category page with no catid (category ID) and with no limit. In other words, each time Joomla encounters a 404 page, it gets all the content items from the database for all the categories! Take a look at this query generated by a 404 page on a Joomla website just before we fixed it:

# Time: 160426 11:49:30
# User@Host: database_user[database_user] @ localhost []
# Query_time: 8.090585 Lock_time: 0.000784 Rows_sent: 76795 Rows_examined: 438986
SET timestamp=1461685770;
SELECT a.id, a.title, a.alias, a.introtext, a.fulltext, a.checked_out, a.checked_out_time, a.catid, a.created, a.created_by, a.created_by_alias, CASE WHEN a.modified = '0000-00-00 00:00:00' THEN a.created ELSE a.modified END as modified, a.modified_by, uam.name as modified_by_name,CASE WHEN a.publish_up = '0000-00-00 00:00:00' THEN a.created ELSE a.publish_up END as publish_up,a.publish_down, a.images, a.urls, a.attribs, a.metadata, a.metakey, a.metadesc, a.access, a.hits,a.state AS state,c.title AS category_title, c.path AS category_route, c.access AS category_access, c.alias AS category_alias,CASE WHEN a.created_by_alias > ' ' THEN a.created_by_alias ELSE ua.name END AS author,ua.email AS author_email,parent.title as parent_title, parent.id as parent_id, parent.path as parent_route, parent.alias as parent_alias,c.published, c.published AS parents_published
FROM #__content AS a
LEFT JOIN #__categories AS c ON c.id = a.catid
LEFT JOIN #__users AS ua ON ua.id = a.created_by
LEFT JOIN #__users AS uam ON uam.id = a.modified_by
LEFT JOIN #__categories as parent ON parent.id = c.parent_id
WHERE a.state = 1 AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2016-04-26 15:49:29') AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2016-04-26 15:49:29')
ORDER BY CASE WHEN a.publish_up = '0000-00-00 00:00:00' THEN a.created ELSE a.publish_up END DESC , a.created;

Not only the query above was causing a huge load issue, but it was also exhausting the maximum memory allocated to each PHP page. So, the 404 page was actually showing the following (instead of, well, a 404 page):

Allowed memory size of 536870912 bytes exhausted (tried to allocate 231 bytes) in /home/[user]/public_html/libraries/joomla/database/driver/mysqli.php on line 811

Of course, when you notice that the above query returned almost 77,000 rows from the database, you start putting things into perspective, and you won’t be surprised by the fact that the whole 512 MB allocated were completely exhausted (by the way, just out of curiosity, we increased the maximum memory in a local php.ini to 2 GB, but still we had the same problem).

So, how do we solve the problem?

Well, fixing this problem is not that hard; you only have to do the following:

  • Open the file category.php located under the components/com_content/models folder.
  • Just after this line:

    $limit = $this->getState('list.limit');

    Add the following lines:

    if (!isset($limit) || empty($limit))
        return array();

  • Save the file and upload it back.

  • That’s it!

As you can see, the whole fix consists of a minor modification to a core file (the modification ensures that Joomla doesn’t query the database if the limit field is not specified), but we understand that some Joomla administrators out there may not want to do this themselves. If you’re one of them, then please contact us, we will do it for you in not time, for an excellent price, and you’ll gain genuine friends and a reliable supplier!

URL Rewriting Not Working on a Joomla Website: How to Fix

We had an interesting job today: a regular client called us in the morning and told us that he can’t get URL rewriting to work on his Joomla 3.5.1 website. He told us that when he set Use URL Rewriting to Yes in the Joomla’s Global Configuration, all the URLs, with the exception of the homepage, resulted in a 404 page.

Naturally, we thought it was a simple issue: most likely the client didn’t rename the htaccess.txt file to .htaccess, but, after a quick check, we discovered that he did, and this is where the fun started…

We first thought that the .htaccess file that he was using was corrupt, and so we replaced it with the default one that comes with Joomla, but that didn’t fix it.

We then thought it might be a system plugin causing all this mess, and so we disabled all the of the enabled system plugins, and we tested the website, hoping that it might be it, but, to our disappointment, it wasn’t: we still saw the dreaded 404 page when we went to anything other than the homepage.

At this point, we started suspecting the server, and so we checked the httpd.conf file (Apache‘s configuration file), but it was clean. We also checked if the mod_rewrite module was installed and enabled, and it was. We tried every single possible trick on the httpd.conf file, restarting Apache every time we did that, but with no success whatsoever.

Since we were so convinced that the problem was a server problem, we contacted the host and we told them that there is some serious issue with URL rewriting on the server, and we would love if they can take a look. They were quick to offer help, but on the condition that we create a very simple test scenario where they can see that URL rewriting is not working. So we created a folder called test under the main Joomla website, and we installed there a fresh, clean copy of Joomla 3.5.1 along with the sample data. We set the Search Engine Friendly URLs and the Use URL Rewriting options in the Global Configuration settings to Yes, we renamed the htaccess.txt file to .htaccess, and we tested the test website. To our astonishment, it worked. This experiment, of course, ruled out the server environment as the cause of this problem, which was somehow depressing: it was our problem again!

Something, however, was weird with the way the website worked when URL Rewriting was enabled. Whenever a rewritten URL was used, the 404 page was loading super quickly, making us question whether Joomla was actually intercepting that page at all. It was a long shot but we wanted to prove it: we added a die(‘Under Maintenance’); at the beginning of the entry index.php file located under the root directory of the Joomla website, and then we visited the URL, and, guess what, we saw the 404 page, instead of a page with just Under Maintenance in it.

This meant that Joomla wasn’t even intercepting the URL, which was odd, as this is not the normal behavior. Now, the question was, where was this 404 page coming from?

It didn’t take us long to discover which file was responsible for generating the 404 page. It was a file called 404.shtml which was located under the main folder of the Joomla website. Renaming that file to somethingelse.shtml fixed the problem!

But why was the presence of the 404.shtml file causing the problem?

The Apache configuration stated that if someone tried to access a file that didn’t exist, and if the 404.shtml file existed, then that file will be loaded. And, since accessing a URL without index.php in it technically means trying to access a non-existent file, the 404.shtml was getting loaded, causing all this frustration for us and for our client.

Now if you, our dear reader, are having a problem with URL rewriting on your Joomla website, then check (after making sure that you have renamed the htaccess.txt to .htaccess) that you don’t have a 404.shtml file lurking on your website. If you do, then just rename it and see if that fixes the problem. If it doesn’t, then please contact us. We will fix the problem for you, efficiently, professionally, and affordably!

How to Submit Data from a HubSpot Form to Your Joomla Website

A client of ours, who has been using HubSpot extensively for a couple of years now, asked us for something fun! He wanted to grab the contact data (when a new form was submitted in HubSpot) in order to create a Joomla user out of it. In other words, whenever someone fills out a specific HubSpot form and clicks Submit, then a new user should be created for him in the Joomla website (out of the submitted form data).

Now we did work with HubSpot a few times before, but this task was more challenging then all of our other HubSpot tasks. We spent days researching how to do this – thinking that this should be done through the HubSpot API, but not finding exactly how. It was frustrating…

But, while searching for a blindfold and a cigarette (we knew we were going to get executed if we didn’t finish this on time), we discovered webhooks. What are webhooks, we hear you ask?

Webhooks are a standard way of passing data from one application to another, easing up the integration of two or more applications. So, in short, you can use a webhook to pass (transmit) the submitted data from HubSpot to the Joomla website (or to any other website/application).

So, how did we do that?

We did it the following way:

  • We logged in to the HubSpot account.
  • We clicked on Contacts -> Workflows in the upper menu.

  • We clicked on Create new workflow on the top right.

  • We entered the workflow name (we chose “Submit to Joomla” as the name, but you can choose any other name), and we chose Standard from the popup window.

  • Under “Automatically enroll contacts when they meet these criteria”, we clicked on Form Submission, and then we chose the HubSpot form that our client wanted to use to register Joomla users from the dropdown, and then we clicked on Done, and then Save.

  • We hovered just over the sentence “Contacts will be enrolled the first time when they meet the criteria above. (Change)”, and, just below it, a link titled “Add action or delay” appeared.

  • We clicked on the “Add action or delay” link, and we selected Trigger a webhook from the dropdown.

  • We chose POST from the first dropdown (you can choose GET if you want to send the information as GET parameters to your Joomla website).

  • We entered the URL of the script that we have already created that will add a user to a Joomla website based on the POST parameters1 (we will not include the script here, but there are many snippets on the Internet on how to do this).

  • We clicked on Save in order to save the webhook and the parent workflow.

  • We activated the workflow by clicking on the Off button next to “Workflow is inactive. Click to review and activate.”, and then, in the popup window, we clicked on Activate workflow now, and then we clicked on OK.

  • We clicked on Content -> Landing Pages in the upper menu, and then we clicked on the landing page where the form resided.

  • We clicked on Edit on the top right, and then clicked on the form (this opened a sidebar to the left).

  • We scrolled down on the left sidebar, and then, under Post submit actions, we clicked on Add to workflow, and then we chose the workflow that we have created above (which we called “Submit to Joomla”).

  • We clicked on Update on the top right.

  • We tested the landing page and we noticed that our script was getting executed when the form was filled! Success!2

If you want to automatically store the data captured by a specific HubSpot form on your Joomla website (or an any other website/application), then try using the above method. If you need help implementing it, then please contact us. Our rates are affordable, our work is professional, and we are the friendliest developers on planet Earth! (or so we say!)

Footnotes:

1HubSpot will not send you the data in the POST headers. It will send it in the body of the headers, and there is a difference. In order to get the form data from HubSpot in your PHP script, you will need to use the following code:

$allHeadersJSon = file_get_contents('php://input');
$allHeaders = json_decode($allHeadersJSon);

$_POST will not work and will not contain any data. You have been warned!

2There is a delay of a few seconds between the form submission on HubSpot and the triggering of the webhook. Keep that in mind if your application is mission critical.

How We Analyze the Logs for Suspicious Logins on a Joomla Website

An important daily task that we perform on the large websites that we maintain is the checking of the IPs that successfully accessed the backend section of the website. We check where these IPs are coming from, and, if they are coming from an unusual location (such as a remote country), then we will look further into it.

So, how do we get the list of IPs that successfully accessed the Joomla backend?

Well, we first password protect the Joomla backend with .htaccess, and we use a unique username in the .htpasswd file, such as websitename_websiteownername. This step is important to do as it allows to know, through the logs, which IPs got past this first authentication.

We then follow the below guide to get the list of IPs:

  • We ssh to the server as root.
  • We change the directory to the logs directory of the website’s user:

    cd /home/[user]/logs/

  • We create a folder called itoctopus (if not already created) under the /home folder:

    mkdir /home/itoctopus

  • We copy the backup file of the current month to the itoctopus folder:

    cp yourjoomlawebsite.com-Mar-2016.gz /home/itoctopus/

  • We change the directory to the itoctopus directory using the following command:

    cd /home/itoctopus

  • We extract the gzipped file that we just copied to the itoctopus folder:

    gunzip yourjoomlawebsite.com-Mar-2016.gz

  • We generate the list of IPs that have accessed the administrator section of the website:

    grep '/administrator/' yourjoomlawebsite.com-Mar-2016 | grep 'websitename_websiteownername' > administrator-access.txt

    (Note: websitename_websiteownername is the user that we created for .htpasswd authentication above and is something like yourjoomlawebsite_jeff.)

  • We get a list of all the unique IPs that have accessed the backed on the Joomla website:

    awk '{ print $1 } ' administrator-access.txt | sort | uniq > administrator-ips.txt

  • We feed the administrator-ips.txt file to a tool that we have built that will automatically generate the locations (city and country) of all the IPs in the list. Note that before building that tool, we used online IP location tools to check where each IP was coming from (which was a somehow tedious process).

  • If we notice that there is an IP that has successfully accessed the website from a suspicious location, then we investigate that IP further (in order to know whether it is a legitimate access or not, and, in the latter case, we check the logs for that IP so that we have an idea on what kind of activities it did on the website).

So, there you go, we have disclosed our method for analyzing logs for suspicious logins on a Joomla website. If you think that the above is a bit too much and you don’t have time to do it yourself, then please contact us. We will do it for you daily, weekly, or monthly based on your requirements, we won’t charge you much, and you will sleep better at night, knowing that access to your website is monitored by Joomla experts (that’s us)!

The Curious Case of Multiple Webmaster Tools Verifications on a Joomla Website

Trivia: The title of this post is inspired by the movie “The Curious Case of Benjamin Button”, which is somehow unique (in our humble opinion), but please don’t unfriend us if you don’t like it. The notion of aging backwards is fascinating, original, and, to some extent, true.

While examining the Joomla website of a new customer of ours this morning, we noticed that there were 3 files and 2 meta tags that were used to verify the site ownership for Google Webmaster Tools (by the way, it seems that Google now calls it the Google Search Console).

Typically, when we see this we know exactly what the deal is: every single developer/designer working on the website added their own Google Webmaster Tools verification code (either by uploading a verification file or by adding a verification meta tag), in order to claim the website’s ownership, so that they can spy on the site’s keywords. The reason why none of them removed the other verification files/meta tags was because they just didn’t know which verification file/meta tag was the legitimate one as they didn’t want to remove it by mistake (so that they wouldn’t get caught).

When this happens, we delete all the verification files/meta tags, and then we ask the client to provide us with his verification file/meta tag (we do prefer having a file over having a meta tag, by the way; it’s just less clutter in the HTML code). Of course, when we tell the client, he is usually annoyed by the behavior of his previous developers/designers, but, on the other hand, he’s exalted because he’s finally dealing with honest developers!

But aren’t there cases where a website has a legitimate need for multiple verifications?

Yes, and that is especially the case of a large website belonging to a very large company. In this case, the company has separate divisions working in silos, and these divisions have separate Google accounts that they use to monitor their site’s performance.

Can this happen with Google Analytics as well?

Yes, and we have seen it many times. The reason why we only spoke about Google Webmaster Tools in this post was because the issue that we saw today was with this particular tool. So, you should also check if your website has more than one Google Analytics verification file/meta tag.

If you have a Joomla website with multiple verification files/meta tags for the exact tool, then you should immediately remove the ones that aren’t yours. If you don’t know how to do that, then please contact us. We will do it for cleanly, quickly, and for very little money. Oh, and we are trustworthy programmers!

Joomla, Pagination, and Performance Issues: Again and Again

Warning: The code changes in this post are in the core. Proceed with caution (and at your own risk) and keep in mind that future Joomla updates may wipe out the below changes.

In a previous post, we have explained how we resolved performance issues caused by pagination on a large Joomla website. The results were satisfactory, but not conclusive: we still had the problem in the backend and we were still seeing the following query (along with some variations of it) in the MySQL slow query log:

SELECT COUNT(*)
FROM myj63_content AS a
LEFT JOIN myj63_users AS uc ON uc.id=a.checked_out
LEFT JOIN myj63_viewlevels AS ag ON ag.id = a.access
LEFT JOIN myj63_categories AS c ON c.id = a.catid
LEFT JOIN myj63_users AS ua ON ua.id = a.created_by
WHERE (a.state = 0 OR a.state = 1);

So, although we solved the issue with excessive pagination, Joomla still was calculating the total items for each particular view, and that calculation was slowing down the whole server. We resolved the issue the following way:

  • We opened the file list.php located under the libraries/legacy/model folder.
  • We changed the following line:

    $page = new JPagination($this->getTotal(), $this->getStart(), $limit);

    to this line:

    $page = new JPagination(2000, $this->getStart(), $limit);

  • We uploaded the file back, we cleared the MySQL slow query log, and then we checked if any new slow queries were written.

  • We waited for an hour and no new query was written to the file. The mission was successful!

But, the next day, we checked the slow query log and we noticed that there were new COUNT(*) queries that were recorded. After a quick investigation we realized that we had to modify another file in order to really close the problem, and that file was the legacy.php located under the libraries/legacy/model folder. Here’s what we did:

  • We opened the aforementioned file.
  • At the very beginning of the _getListCount function, we added the following line:

    return 2000;.

  • We saved the file and we uploaded it back.

  • The problem was solved!

Now, you might be wondering, what’s with the 2000? Well, the 2000 is a hardcoded number representing the number of entries in any view. For example, if a category has 10,000 articles, then we are telling Joomla that it has only 2,000 articles, which is OK (because we are not allowing pagination past page 100). But, what if a category only had 10 articles? This means that for that category, we will be showing that it has 100 pages when it only has 1 page. That doesn’t look very nice, but it cannot be handled with a single line of code. So, if you want to implement the solution above, you will need to live with this limitation, or you will need to ask some Joomla experts (such as your humble servants), to address this for you. If you want to do this yourself, then let us give you a hint: you will need to run a midnight cron job calculating the number of items for each view, and, if that number is less than 2000, then you should use that number in the files above instead of 2000.

If you want help implementing the above solution, then please contact us. We are happy to serve, we work very quickly, and our fees are very affordable!

Is Joomla 3.5 Really 50% Faster than Joomla 3.4.8?

Rumors spread around quickly, and the latest rumor, besides the one that states that the world is not flat, is that Joomla 3.5 is 50% faster than Joomla 3.4.8. To be more specific, the rumor states that Joomla 3.5 is 50% faster because, unlike Joomla 3.4.8, it can run under PHP 7. So, the rumor can be traced down to another rumor which states that PHP 7 is 50% faster than PHP 5.

So, is it true that PHP 7 is 50% faster than PHP 5?

Zend’s website proclaims that it is, more or less, true. In fact, they (Zend – the developers behind PHP) say that Drupal is 72% faster under PHP 7, and Magento‘s catalogue is over 100% faster (we know, where is Joomla, huh?). If you’re thinking that this is too good to be true, then you are probably right: the scripting language (PHP is a scripting language) is very, very rarely the bottleneck on any website. In 99.99999% (that’s five decimal nines) of all cases, the bottlneck is the database server. So, any improvement at the scripting language’s level is always negligible since a Joomla page (or a page on any other CMS) has to wait on the database server before it is served to the client.

Of course, the above assumes that Zend’s hype is not merely a propaganda – but this is not really the case. In fact, most of those who tested PHP 7 on any website are claiming that they are getting worse results than with PHP 5, which doesn’t necessarily mean that the Zend team is lying with statistics, but it definitely means that all Zend tests were run in optimal environments where PHP 7 excels.

But what if PHP 7 is really 50% faster than PHP 5?

As stated above, the bottleneck of any CMS is the database, so even if PHP 7 is ten times (1000%) faster than PHP 5, nobody will feel it.

Of course, there are some Joomla administrators out there who are claiming that their websites are flying with Joomla 3.5 and PHP 7, but we think that this is a case of the emperor has no clothes, and it’s only a matter of time before everyone becomes aware of the fact that it’s all a hype, and that Joomla 3.5 is not even slightly faster than Joomla 3.4.8 (simply because the necessary database optimizations were not implemented).

So, if you’re excited about upgrading to Joomla 3.5 because you’re thinking that it can resolve all (or even some) of your performance issues, then think again, because your performance issues are almost invariably caused by MySQL and not PHP. If this post made you hesitate a bit and you want to seek some advice before updating to Joomla 3.5, then please contact us. Note that our super affordable fees apply.

A Quick Shell Command to Check the List of Modified Files on a Joomla Website

One of the things that we do daily on the websites that we fully manage is check which files were changed in the past 48 hours. Doing this helps us catch files that were maliciously modified, it also helps us check whether the client’s employees have uploaded files that they shouldn’t have uploaded, or have uploaded files to the wrong folder.

So, how do we do that?

There are several ways to do that, but we prefer to do this through root shell access. So, we ssh to the server, and then we run the following command:

find /home/user/public_html -path '/home/user/public_html/cache/*' -prune -o -mtime -2 -ls > changed.txt

The above command will dump the list of modified files within the past 48 hours in the changed.txt file under the /home/user/public_html (it excludes files created under the cache folder). Once we run the above command, we then check each and every created and modified file. For example, if we see that a PHP file was created or modified, we check that PHP file immediately: PHP files should never be modified, and they should only be created when a new extension is installed, other than that any modification/creation of a PHP file is most likely malicious.

If we see that a new image file was created, we check if it was created under the right directory (we ensure that images are evenly distributed under the images folder).

How to exclude certain file types from being included in the list of changed files?

Many Joomla website owners don’t care about the organization of the image files the way we do, so they might not be interested in knowing which images files were changed, in that case, they can run the following command to generate a list of changed files, but excluding image/PDF files:

find /home/user/public_html -path '/home/user/public_html/cache/*' -prune -o -mtime -2 -ls | grep -vE '(.jpg|.png|.gif|pdf)' > changed.txt

The above command generates a list of all the files created/changed within past 48 hours with the exception of image files and PDF files.

But, what if you want to get a list of changed files that were modified in the past week?

Some Joomla administrators run weekly checks on their Joomla websites (they don’t have time to run daily checks), so they might want to go with 7 days instead of 48 hours. In that case, all they need to do is change 2 in the above code to 7.

The above is one of the many checks we run daily on the Joomla websites that we fully manage. If you want us to fully manage your website, please contact us. Our fees are affordable, our work is professional, and we will proactively ensure that your website is clean!

How to Resolve Performance Issues Caused by Pagination on Large Joomla Websites

Warning: The solution presented in this post consists of a core modification, which means a future Joomla update may wipe it out. You have been warned!

If you check the MySQL slow query log of a large Joomla website, you will probably see something like the below query:

SELECT a.id, a.title, a.alias, a.introtext, a.fulltext, a.checked_out, a.checked_out_time, a.catid, a.created, a.created_by, a.created_by_alias, CASE WHEN a.modified = '0000-00-00 00:00:00' THEN a.created ELSE a.modified END as modified, a.modified_by, uam.name as modified_by_name,CASE WHEN a.publish_up = '0000-00-00 00:00:00' THEN a.created ELSE a.publish_up END as publish_up,a.publish_down, a.images, a.urls, a.attribs, a.metadata, a.metakey, a.metadesc, a.access, a.hits, a.xreference, a.featured, a.language, LENGTH(a.fulltext) AS readmore,CASE WHEN badcats.id is not null THEN 0 ELSE a.state END AS state,c.title AS category_title, c.path AS category_route, c.access AS category_access, c.alias AS category_alias,CASE WHEN a.created_by_alias > ' ' THEN a.created_by_alias ELSE ua.name END AS author,ua.email AS author_email,parent.title as parent_title, parent.id as parent_id, parent.path as parent_route, parent.alias as parent_alias,ROUND(v.rating_sum / v.rating_count, 0) AS rating, v.rating_count as rating_count,c.published, CASE WHEN badcats.id is null THEN c.published ELSE 0 END AS parents_published
FROM #__content AS a
LEFT JOIN #__content_frontpage AS fp ON fp.content_id = a.id
LEFT JOIN #__categories AS c ON c.id = a.catid
LEFT JOIN #__users AS ua ON ua.id = a.created_by
LEFT JOIN #__users AS uam ON uam.id = a.modified_by
LEFT JOIN #__categories as parent ON parent.id = c.parent_id
LEFT JOIN #__content_rating AS v ON a.id = v.content_id
LEFT OUTER JOIN (SELECT cat.id as id FROM #__categories AS cat JOIN #__categories AS parent ON cat.lft BETWEEN parent.lft AND parent.rgt WHERE parent.extension = 'com_content' AND parent.published != 1 GROUP BY cat.id ) AS badcats ON badcats.id = c.id
WHERE a.access IN (1,1) AND c.access IN (1,1) AND CASE WHEN badcats.id is null THEN a.state ELSE 0 END = 1 AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2016-03-13 13:01:11') AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2016-03-13 13:01:11')
ORDER BY c.lft, a.featured DESC, fp.ordering, CASE WHEN a.publish_up = '0000-00-00 00:00:00' THEN a.created ELSE a.publish_up END DESC , a.created LIMIT 34680, 20;

Now we have explained how to optimize most of this query a long time ago, but still, there is one part of it that we didn’t address, it’s the LIMIT 34680, 20 part, in which we tell MySQL to grab results which ordering is between 34680 and 34679 (inclusive) for pagination purposes.

The problem with the LIMIT 34680, 20 is that it is extremely slow when fetching results, and can cause a delay of 5-6 seconds.

So, what’s the solution?

Well, a LIMIT 34680, 20 means that we are trying to fetch the data for page 1735 (since we are listing 20 results per page, so that’s 34680/20 + 1), but that it’s a bit too excessive. Even Google, with all its processing power, doesn’t do that. If you don’t believe that, try visiting the following link: https://www.google.ca/?gws_rd=ssl#q=test&start=3000, and you will see the below message:

Sorry, Google does not serve more than 1000 results for any query. (You asked for results starting from 3000.)

So, Google, the most important website in the world, restricts the number of its search results to 1000 results (or 100 pages, since they display 10 results per page), but we are trying to display more than 30,000 results on the Joomla website, which doesn’t make much sense.

So, we discussed the issue with the client and we convinced him that it’s a much better idea (for performance reasons) to restrict pagination to only 100 pages (2,000 results, that’s 1,000 more than what Google displays), and the client immediately agreed, and so implemented the fix the following way:

  • We opened the file index.php located under the main directory of the website.
  • We added the following code at the very beginning of the file:

    $myStart = intval($_GET['start']);
    if ($myStart > 2000)
    	die('Pagination Limit Exceeded');

  • That’s it. The problem was solved! No more slow queries caused by excessive pagination!

But wouldn’t the above cause a problem when indexing older articles?

Usually, humans (unless they are really, really, really bored) never browse past page 10, only bots do, and they only do that to index older articles. If you disallow pagination past page 100, then you might run into indexing issues. In order to resolve the problem, you will need to make sure that you have a valid and up-to-date XML sitemap of your website.

Isn’t there a better solution?

Well, if you have contiguous IDs on your Joomla website, then the answer is yes, there is a better solution that allows you to have all the pagination that you want, and it consists of changing the LIMIT statement to something like WHERE id > n LIMIT 0, 20 (where n substitutes 34680 in the query above). However, on large Joomla websites, it’s almost an impossibility to have contiguous IDs.

If you have a large website, we really recommend implementing the above solution. It will definitely lessen the load on your server and will make your Joomla website work a lot faster. If you’re afraid of the implementation, then please contact us. We will do it for you quickly, professionally, and for a very reasonable fee!

Joomla’s Backend Displaying phpMyAdmin

We had an interesting case today – a client of ours told us that whenever he tries to login to the administrator section of his website, he was seeing phpMyAdmin‘s login page instead of Joomla’s regular administrator login page. At first, we thought we misunderstood what the client said, but it didn’t take us long to discover that we didn’t (yes, there are 2 “didn’t’s” in this sentence, but it’s not a double negative).

We saw weird things on Joomla websites, but it was the first time we saw something that weird. What was even weirder was that the administrator folder of the Joomla website contained an extracted copy of phpMyAdmin (hence the phpMyAdmin login). It still contained everything the administrator folder had (with the exception of the index.php file, which was overwritten by that of the phpMyAdmin instance). Fixing the problem consisted of simply copying the administrator/index.php file from a fresh Joomla install (matching the client’s version) to the administrator folder of the actual website. Of course, we needed also to remove the folders associated with phpMyAdmin, but that wasn’t necessary to fix the problem.

But what caused this to happen?

We have 2 theories on what caused this:

  1. A staff member (working for the client) or the hosting company mistakenly uploaded phpMyAdmin to the administrator folder or our client’s website.
  2. The website was hacked, and the phpMyAdmin instance was uploaded to the administrator folder as part of the hack.

Obviously, we can’t do anything to avoid the first potential cause in the future as it is a human mistake. But for the second one, we implemented some security measures on the website (such as preventing access to files other than the index.php file) in order to better protect the website.

If your Joomla’s backend is suddenly displaying phpMyAdmin (or a different application), then make sure that phpMyAdmin (or that other application) is not installed in your administrator folder. If it is, then all you need to do is to remove it (e.g. remove the application) and restore the original administrator/index.php file. If that doesn’t solve the problem, then please contact us. We will solve the problem for you in as little time and for as little money as possible.

Joomla Website Crashing Constantly After Moving to a Larger Hard Disk

A regular client called us a week ago and told us that, after moving his Joomla website to a larger hard drive, it started crashing constantly. We checked the load on his server and it was super high (in the double digits) which explained the constant crashing, but didn’t really reveal the root cause.

So, we checked the slow query log (since the process that was creating the majority of the load was MySQL) and we noticed that there was a huge number of various slow queries, which was odd, considering that the website was running very smoothly before the move to a larger hard drive. Naturally, we started optimizing the queries, one by one, but we just couldn’t get the website under control.

We then reduced Bing’s crawl rate, as well as Google’s (we did the latter task using Google’s Webmaster Tools). Things got better, and the website crashed less often, but it still crashed…

We then did another round of optimization, and we blocked all the spam and the non-essential bots, but the progress wasn’t substantial and the load was still un-maintainable.

We then remarked that around midnight, a cPanel process called pure-quotecheck was causing some additional load (second highest load after MySQL), and so we stopped it by disabling updates in WHM. The load decreased, but still, it wasn’t a silver bullet.

We literally worked for a week trying to optimize the website in order to get it back to stability, but we weren’t able to. We were ready to declare defeat, but, in a moment of despair, we re-read the first email that the client sent, which stated that the whole problem happened after the website was moved to a larger hard disk. So we thought, what if the new hard disk was not an SSD (Solid State Drive) hard disk (the previous hard disk was an SSD hard disk)? To our surprise, a quick test revealed that the new drive was a SATA (Serial AT Attachment) drive, which is around ten times slower than SSD! Aha! We found the root cause of the problem!

Immediately, we asked the hosting company to switch our client back to an SSD drive, which they managed to accomplish in a few hours, and, once they did, the website stabilized and became super fast (even faster than before the move because of all the optimizations that we did).

But how come we didn’t think of that before?

Our client communicated heavily with the host about this problem before resorting to us, so we couldn’t really imagine that it’s a hardware issue (although we did start suspecting that the hard disk was faulty near the end of our investigation, just before we discovered that it was a SATA drive). And honestly, we never thought that the host would do such a thing by mistake (moving a website from SSD to SATA).

But why would a switch from SSD to SATA have such terrible effects on a Joomla website?

Well, on large Joomla websites, MySQL tends to create a lot of temporary tables on the file system. As mentioned above, SATA drives are usually about 10 times slower than SSD drives when it comes to writing data, and so, on SATA drives, MySQL faces a bottleneck when writing those temporary tables. This leads to a huge load caused by MySQL waiting for the SATA drive to write the temporary tables. The problem, of course, is exacerbated as additional MySQL threads try to write some temporary tables, and have to wait in line until the previous threads are served by the SATA drive.

So, our dear reader, if your website starts crashing after moving to a larger hard disk, then make sure you were not moved, by mistake, from an SSD drive to a SATA drive. If that was the case, then ask your host to move you back to SSD. If that wasn’t the case, then please contact us. We always find a solution, our prices are affordable, and we are the friendliest developers on planet Earth.

Internal Server Error When You Use PHP 5.5 for Your Joomla 3.x Website

For the somewhat technically savvy, the title of this post may seem like an oxymoron: Joomla 3.x should run very well on any PHP version ranging from 5.3.10 through version 7! So why should it have a problem with PHP 5.5 (or higher versions of PHP)?

To answer this question, let us tell you a little story…

Around midnight, a client emailed us and told us that he can’t run his Joomla 3.4.8 website on PHP 5.5 (which is his host’s default), and that he has to add the following line to his .htaccess file in order to make his Joomla website work:

AddHandler application/x-httpd-php53 .php

The above code ensured that the PHP version used to power his Joomla website was 5.3.29, which worked for our client, but he didn’t (and rightly so) want to remain on a no longer supported PHP version.

The first thing that we did when we started working on the website was removing the above line from the .htaccess file, and the moment we did that, the website displayed the infamous Internal Server Error page. The client, in his email, told us that the problem was caused by an extension called HotSpots which (again, according to our client) did not work (for some reason) on PHP 5.5. So, we disabled that extension, and still, we had the same problem.

We then disabled all the 3rd party extensions, one by one, until we were left with only the core extensions (we even switched to a core template), but still, we had the Internal Server Error page on the frontend.

Naturally, we checked the Apache error log, which revealed nothing, since the client was on a shared hosting. We expected though to get the server logs from the host, but they (the host) refused to share them us, so we had to fix the problem ourselves.

We then started debugging the entry index.php file: we started, as usual, by adding a die(‘Under Maintenance); code to its very beginning. To our surprise, the Internal Server Error remained even after adding the die line to the beginning of the index.php file. So, we thought it was a problem with the file permission, but, unfortunately, it wasn’t (the permissions were set to 644 on the index.php file, and the file ownership was correct). So, we just thought that this file was jinxed, and we created another file, called test.php which contained the following line:

<?php die('Under Maintenance'); ?>

Unsurprisingly, when we tried to load the test.php file from the browser, we were greeted (again) with the Internal Server Error page.

At this point we started suspecting the host: perhaps the host’s PHP 5.5 version is corrupt, and all their clients think that the problem is on their end, and so they add that .htaccess line to run their websites under PHP 5.3, but, we then quickly dismissed that idea since we had another client on the same host who didn’t have this problem.

Only after hours of searching that we discovered the root cause of the problem: there was a php.ini file, placed under the root directory of the Joomla website, which was only compatible with PHP 5.3. Renaming the php.ini file to php.ini.old immediately solved the problem!

If you are seeing an Internal Server Error on your Joomla website when you are using PHP 5.5, then check the root directory for a php.ini file (or a .user.ini file), if you have one, then just rename it to php.ini.old (or .user.ini.old) and see if that fixes the problem. If it does, then you’re welcome! If it doesn’t, then please contact us, we’ll solve the problem for you in very little time and for very little money.

How to Disable Browser Caching in Joomla’s Backend

We received the following email from a client yesterday:

“I’ve had a problem for about a year with my company website. It makes me attempt the login 3-5 times before getting into the backend, and then when I’m in, I have to save any change I make to the site either 2, 3, or 4 times. I just dealt with this weird issue for a while but I can’t take it anymore. I don’t have any coding or technical experience (I’m a long-term amateur Joomla guy). I’m wondering if you guys could help.”

We were perplexed, because we really haven’t seen anything like this before, except maybe for this, but it really didn’t apply to our client since he wasn’t being redirected from www to non-www (or vice versa) when he tried to login and the problem was also happening when he was trying to save articles (or any content item, as we later discovered).

So, we started testing the website and it was very annoying, every time we tried to login we had to login multiple times. The first time it was showing a blank page, and the second time it was showing the following error:

The most recent request was denied because it contained an invalid security token. Please refresh the page and try again.

The third time the website displayed the above error again, and then, when we went to the login page the fourth time (or the fifth time), the website logged us in immediately (without even having to re-enter the username and password).

Now, of course, we wanted to know why the website was showing a blank page the first time we tried to login… So, we set error reporting to Maximum in the configuration.php file and then we tried to login again, and this time, the first time we tried to login, we saw the following error:

Fatal error: Call to a member function checkSession() on a non-object in plugins/user/joomla/joomla.php on line 216

A quick research on the above error revealed that the cause of the problem was the presence of the folder libraries/joomla/plugin folder, which was there from a previous Joomla version, and so we deleted the libraries/joomla/plugin folder, thinking that this would solve the main problem. But it didn’t, it just redirected us to the login page the first time we tried to login (the second time [and the third time, and the fourth time] it still showed the error “The most recent request was denied…”), and so we continued testing and experimenting.

We then remarked something: when we modified an article in the backend, it was modified immediately in the database, but it was still loading the old article in the backend, which meant that the issue was caused by an aggressive type of browser caching. Aha!

So, in order to fix the problem, we instructed the browser not to cache the pages on the Joomla website by adding the following to the very beginning of the .htaccess file (located under the main directory of the website, but we could also have created a different .htaccess file and placed it under the administrator folder):

<FilesMatch "\.(html|htm|php)>
FileETag None
Header unset ETag
Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate"
Header set Pragma "no-cache"
Header set Expires "Mon, 01 Jan 1990 00:00:00 GMT"
</FilesMatch>

As soon as we added the above code, everything worked normally: we only had to login once and we only had to save content items once. The problem was solved! Hooray!

But what caused that aggressive browser caching in the first place?

The website was on a shared hosting, so we assume that the host has enabled this aggressive browser caching at the Apache level in order to reduce the server’s overall traffic (and load).

Now, if you have to login multiple times to your Joomla website and/or you have to save the same article multiple times until you see your changes in the backend, then try adding the above code to your .htaccess file to disable browser caching. If the problem persists, then please contact us. We’ll be happy to help, we always find a solution, and our fees are super affordable!

Google News Not Displaying Images for Joomla Articles: How to Fix

A high traffic law news website that we have worked on many times had a long-time problem with Google News – a problem that we were made aware of a few weeks ago. It wasn’t a serious problem, but it was quite annoying: whenever they posted a new article on the website, Google News would index the article, but would not show the article’s image in its listings. Now, of course, this isn’t catastrophic, but this problem reduced the traffic directed to this website (people are more willing to click on articles with images) and, of course, made the website look slightly unprofessional.

Now, any developer would agree with us that the worst website problems to solve are those that cannot be directly controlled from within the website. And that problem was exactly that – no one has control over what Google decides to show and how it indexes websites. Of course, there was a problem on the website, but we just didn’t know what it was because it wasn’t actually happening on the website.

So, what did we do?

We tried everything. And when we say everything, we mean everything. We read Google News‘ technical requirements (available here) many times, many many times, but with no avail: the image simply wasn’t displaying – instead – the alt attribute of the image was displaying.

What made things even more complex is that we didn’t know whether our fixes worked or not immediately as we had to wait until Google indexes the website. We also couldn’t just add test articles to the website (and then delete them) because of its importance in the legal world (the team managing the website had zero tolerance for on-site debugging).

We were about to declare defeat, total defeat, when a lightning bolt struck us (well, not literally as there aren’t lightnings this time of year in Montreal) after re-reading the technical requirements for the gazillionth time (we are slightly exaggerating here, we only read it about 2 billion times), where we noticed this little sentence about the robots.txt file: “Please read our Help Center article about robots if you believe your site’s robots.txt file, meta tags, or HTML header specifications might be blocking our crawler from accessing your content.” Well, what if it was the robots.txt file?

So, we checked the path of the image of a random article on the website, and we noticed that the path was something like: http://www.ourclientjoomlawebsite.com/cache/multithumb_thumbs/image_file_name.jpg (the client uses the BK-MultiThumb plugin for generating thumbnail images).

But, Joomla’s default robots.txt clearly instructs search engines not to index anything in the cache folder! That’s it! That is really it! Google News wasn’t displaying the images because we told it not to in the robots.txt file! So to fix the problem, all we had to do was to tell Google to index the cache/multithumb_thumbs folder by adding the following line to the robots.txt file immediately after Disallow: /cache/:

Allow: /cache/multithumb_thumbs/

Phew! That simple line fixed a year-old problem!

It really took us a long, long time to get to the bottom of this, and we are glad that we have shared our knowledge in this post so that other people with the same problem will be able to easily fix it. But, if even after following the instructions above, you are still having problems with Google News not displaying your article images, then please contact us. We’re always happy to help, we love solving problems, and our rates are super affordable!

Email Sent but Not Received When Emailing Articles on a Joomla Website

Note: The solution presented in this post consists of a core modification. Core modifications may be wiped out with future Joomla updates. Please keep that in mind should you decide to implement it.

Last Thursday, a regular reader of a client’s website emailed our client and told them that whenever he likes an article on their website, he tries to email its link to himself for future re-reading, and he does that using the “Send this article to a friend” functionality. The problem is, he rarely, if ever, receives the email containing the link to the article (despite the site telling him that the “Email was sent successfully”), so he ends up doing the whole process manually (he copies and pastes the link, and then uses his own email tool to send the link to himself). The reader explained in his email that he was using Comcast.

Now we did have a problem before with this particular functionality (sending articles to a friend), but that problem was addressed on this particular website, and, in addition, the Joomla application clearly stated that the email was successfully sent when the current problem happened, which wasn’t the case for the older problem (which, again, was resolved on this particular website)…

So, we tested this feature by trying to email an article to our email, and, although we received the email, we noticed one thing: the “from” email address was the “Email” field in the popup form, which meant that technically, the Joomla website was trying to send emails on behalf of other domains when using that functionality. We immediately knew that the problem was related to SPF (see this post on why SPF can cause problems when sending emails).

In order to fix the problem we had to make sure that Joomla sends emails on behalf of a domain that is authorized to send emails from the website’s (the Joomla website) IP. Typically, that domain is the actual domain of the Joomla website. We did it this way:

  • We opened the file “controller.php” located under the /home/hospital/public_html/components/com_mailto folder.
  • We changed the following line:

    if (JFactory::getMailer()->sendMail($from, $sender, $email, $subject, $body) !== true)

    to:

    $siteName = JFactory::getConfig()->get( 'sitename' );
    $siteFrom = $config->get( 'mailfrom' );
    $siteSender= $siteName.' on Behalf of '.$sender;
    if (JFactory::getMailer()->sendMail($siteFrom, $siteSender, $email, $subject, $body) !== true)

  • We emailed the user and asked him to try to send an article to himself. He responded immediately with a very sweet “Thank you! It’s working!”.

But what if after applying the fix above the problem still occurred?

In that case, then most likely the website’s IP is not allowed to send emails on behalf of the website’s domain. To fix this problem, you will need to make sure that you add the website’s IP to the domains’s SPF setting (we explained how to do that in the second link above).

But, how come we didn’t have this problem when we tested it?

Some domains, including the domain that we use for testing, are hosted on servers that are lax in enforcing spam rules, and that will allow almost all emails to go through, including emails that are sent from IPs that are not authorized to send them.

Now, if you hare having problems emailing articles on your Joomla website, try applying the above fix. If it doesn’t work, then please contact us. We will fix the problem for you as quickly as humanly possible, we will then ensure that it’s working all the time, and finally, we will send a very reasonable invoice that will incite you to work with us again!

The “Advanced Mode” in Joomla’s Redirect Manager: A Mystery Unveiled

When you start delving into the wonderful world of Joomla, you will soon know that it has many mysteries; some of which you are curious to know everything about, others of which you prefer to leave alone. In the latter category, we can think of one mystery, the Redirect Manager‘s Advanced Mode. Before explaining what is that, let us tell you first where to find it!

You can find this option by logging in to the backend, and then clicking on System -> Global Configuration, and then clicking on Redirect on the left tab. You will see the Activate Advanced Mode option (which defaults to “No”) under the Advanced tab on the top.

So, what is it?

Frankly, we have never used it and we didn’t know what it was, until earlier this week, when a client of ours drew our attention to it, asking us what it does. So, we checked the code, mainly in the Redirect Manager plugin (e.g. in the file redirect.php located under the plugins/system/redirect folder), and we saw this:

// If no header is set use a 301 permanent redirect
if (!$link->header || JComponentHelper::getParams('com_redirect')->get('mode', 0) == false)
{
	$link->header = 301;
}

In case you haven’t guessed it already the mode attribute represents the Advanced Mode, and is defaulted to zero (“No”). The default behavior (when the Advanced Mode is set to “No”) is that the HTTP header of any link is 301 (which means that the link is redirected permanently to a different link [the Destination URL]).

Now, what happens if it’s set to “Yes”?

Well, in that case, the stored HTTP header of the link will be used, so, if it’s something like 303 or 304, then it’ll be used instead, instead of the default 301 redirect.

Now, you might be wondering, how does Joomla know which link is a 404 (not found) link, for example? Well, manually, of course! You see, when you switch the Advanced Mode to “Yes”, you will be able to specify the Redirect Status Code for each link (in the Redirect Manager component). Oh, and yes, the Destination URL will no longer be mandatory once you set the Advanced Mode to “Yes”, since some HTTP codes (e.g. the Redirect Status Codes), do not entail a redirection to another link.

In case you’re wondering how the “New Redirects” page will look like once you set the Advanced Mode to “Yes”, then here it is:

New Redirect Page with Advanced Mode Set to Yes

Figure 1: New Redirects Page with Advanced Mode Set to “Yes”

So, if you were wondering about that mystery, then we hope that we have unveiled it for you. If you think we haven’t, or if you need further explanation on the subject, or if you want us to unveil other Joomla mysteries for you, then please contact us. We know Joomla inside out, we respond quickly, and our fees are super affordable!

How We Ran an SSH Command from Joomla

Note: This post is very advanced, and requires some Linux shell experience. If you feel that your experience is not up to the suggestions below, then we suggest that you ask some experts to do it for you.

Another note: Please read the post to the end before performing any work.

Recently, Apache‘s ModSecurity has become quite aggressive in blocking suspicious requests. Most of our major clients are finding its new behavior (ModSecurity‘s behavior) annoying, since it’s often blocking their IPs when they inadvertently trigger a security rule. Whenever that happens, they call us and we will need to unblock them by ssh’ing to the server and running the following commands:

csf -dr [our-client-ip]
csf -r

The first command drops (hence the dr) from the CSF database, and the second command refreshes CSF and is needed to re-allow access for the blocked IP.

But what is CSF?

CSF, in case you’re wondering, stands for ConfigServer Security & Firewall, and it’s the tool that effectively does the IP blocking (often at the request of ModSecurity, but it has its own blocking rules as well, such as when an IP tries to FTP with the wrong password, or tries to access an htpasswd protected area with the wrong password). CSF is installed by default on almost all Linux servers, as it is critical for any server’s security and stability.

But what is the problem with the above?

We typically process the unblock requests instantly, and by instantly we mean by the time we get the email, which can take anything between a minute and 15 minutes, and during those minutes the client can’t access their own website from their network. Not good!

So, while the above process works, it is not ideal for the client since it makes us the bottleneck, and clients aren’t very fond of that. We had to develop a method that allowed our major clients to unblock themselves from within the Joomla website. But there were 2 challenges:

  1. If they are blocked at the server level, then how can they access the Joomla backend to unblock themselves?
  2. csf unblocking commands require root access for successful execution, which means that we have to run the ssh2_exec PHP function, which means that we have to install the libssh2 PHP library, which may pose a major security threat to the server.

So, how did we overcome the issues above?

The first issue was easy to address – all that we needed to do was to ask the client to ask a team member to access the website using his mobile data connection, which typically uses a different, unblocked IP.

The second issue was trickier: first we needed to install the libssh2 on the server (and this is not a straightforward task, take our word for it), and then we had to create the following script:

$ourClientIP = $_POST['ourClientIP']; //The client IP to unblock is submitted using a simple form
if (!filter_var($ourClientIP, FILTER_VALIDATE_IP)) {
	die('Invalid IP');
}
$sshConnection = ssh2_connect('ourclientjoomlawebsite.com', 22);  //22 is the default ssh port, many servers elect to use a different port for security reasons
ssh2_auth_password($sshConnection , 'root', 'rootPassword');
ssh2_exec($sshConnection , 'csf -dr '.$ourClientIP);
ssh2_exec($sshConnection , 'csf -r');

The above code is fine and dandy, but there is one problem with it, one super-major problem: we are exposing the root password in a PHP script file, which means that any developer with FTP access to the Joomla website will be able to know what the root password is. Additionally, if the Joomla website has any exploit, then a malicious user will gain access to the script, and will be able to know what the root password is.

But, we are always ingenious (and humble!), and so we devised a way that allowed our client to unblock their IP without compromising server’s security. Here’s how:

  • We created a simple (htpasswd protected) form where the user enters the IP to unblock. Once the IP is entered, we save it in a text file called unblock.txt, which is located under the administrator folder.
  • In ssh, we ran the following command:

    crontab -e

    and then we added the following line to the very end (and then saved the cron):

    * * * * * if ls /home/[user]/public_html/administrator/unblock.txt > /dev/null 2>&1; then value=`cat /home/[user]/public_html/administrator/unblock.txt`;/usr/sbin/csf -g "$value";/usr/sbin/csf -dr "$value";/usr/sbin/csf -r;rm -f /home/[user]/public_html/administrator/unblock.txt; fi

The above 1-minute cron command does the following: 1) it grabs the IP to unblock from the blocked.txt file, 2) it checks if the IP already exists in the CSF database (unnecessary step), 3) it then removes the IP, and, 4) it finally refreshes the CSF database and removes the unblock.txt file. Of course, these steps are only performed when the unblock.txt file exists.

By implementing this simple logic in the cron, we avoided installing an additional PHP library and, much more importantly, we ensured that the root password remained “secret”.

Now, if you want to run a different ssh command from your Joomla website, then just follow the instructions above, it will work! If it doesn’t, then please contact us. We will help you do this in no time and for a super affordable fee!

K2 Feed Not Working when sh404SEF Is Enabled – How to Fix

We had a curious case last week. A client called us and told us that his website’s K2 RSS feed was not working. He was using FeedBurner, and he told us that it (FeedBurner) was displaying content that was weeks old instead of displaying the fresh content from his website.

So, we checked the website link that FeedBurner was using to retrieve the feed and we noticed that it was redirecting to the corresponding page on the Joomla website, instead of the RSS page. That was odd, and we have never seen this before.

Now, what was special about this website is that it was using sh404SEF, which is an extension that is known to cause many issues. So, we looked up the non-SEF link of the feed in sh404SEF‘s URL Manager, and we found that it was something like: ourclientjoomlawebsite.com/index.php?option=com_k2&Itemid=178&format=feed&id=55&lang=en&layout=category&task=category&view=itemlist. We then disabled sh40SEF, and we tried to load the non-SEF link, and, to our surprise, it worked! It displayed the actual RSS feed instead of redirecting to the normal view.

The thing is, the fact that it worked without sh404SEF made things a bit more confusing (despite the fact that we know that the culprit is sh404SEF), because it just didn’t make sense since sh404SEF should just rewrite the URL, and nothing more. We then spent many hours trying to find the cause of the problem, and we were close to declaring total defeat, but then, something happened: our client emailed us and told us that he has another extremely similar website, where the feed was working (that website had the exact same extensions and template installed – it was technically the same website but used another domain, protected by an .htpasswd, and was used for testing purposes).

We compared the 2 sites together and the only difference that we noticed was that on the development website (the website that didn’t have the problem), the following global configuration settings were all set to “No”:

  • Search Engine Friendly URLs
  • Use URL Rewriting
  • Adds Suffix to URL

While on the actual, production website, the above settings were all set to Yes. So, we changed those settings on the production website to No, and, guess what? It worked! The problem was fixed!

But why did the above settings cause the problem?

We think this was because of a conflict between sh404SEF and Joomla’s own SEF. Typically, these two play well together but, in that particular scenario, they didn’t. We’re not sure of the exact cause because we didn’t do a thorough investigation.

If you have the same problem on your Joomla website, then try disabling the above settings in the global configuration, and see if that fixes the problem. If it doesn’t, then just contact us! We will find a solution (we always do), we will implement that solution, and we will not charge you much!

Warning: The Similar URLs Plugin Will Slow Down Your Joomla Website

If you’re an avid reader of our blog, then you may already know that we’re not huge fans of sh404SEF. In fact, we have gained expertise in getting rid of it on large websites. So when something wrong happens to a website that has sh40SEF installed, we immediately suspect this notorious extension, and usually, we are not proven wrong!

Let us give you an example… Late last week a client came to us and told us that they were having some serious performance issues on their Joomla website. We knew that they were using sh404SEF and so we disabled the whole extension in order to confirm whether the problem was caused by sh404SEF or not. And, again, our suspicions were not wrong: the website worked at optimal speed once sh404SEF was disabled.

But, the client wanted sh404SEF and wasn’t ready to get rid of it (getting rid of sh404SEF on large websites is a painful project), and so he asked us to see what we can do about it. So, we checked the slow query log and we noticed that it was full of something like the following query:

select oldurl, newurl, id, dateadd from #__sh404sef_urls where newurl <> “” AND oldurl not like ‘%vmchk%’ AND newurl not like ‘%format=feed%’ AND newurl not like ‘%format=pdf%’ AND newurl not like ‘%print=1%’ AND oldurl not like ‘%__404__%’ AND ( soundex(oldurl) = soundex(‘rss/feed.html’) OR oldurl like ‘%rss%’ OR oldurl like ‘%feed%’) GROUP BY oldurl limit 500;

A quick search in the codebase revealed that these slow queries stemmed from the sh404SEF’s Similar URLs system plugin. So, in order to address the performance issue, all we needed to do was to disable that plugin.

But, doesn’t disabling the plugin have any negative repercussions on the website?

No – it doesn’t. In fact, what this plugin does is that it tries to redirect a URL with a small error (such as a typo) in it to the right URL. Now this is all fine and dandy and it sounds great on paper, but in practice, it is completely a different issue. First, it doesn’t work most of the times and second, when it does work, it can cause SEO issues (any respectable search engine out there doesn’t like to see dozens of links for the same article). So, disabling this plugin is in fact a duty for any Joomla administrator out there, since it hurts both performance and SEO.

If you have the Similar URLs plugin enabled on your Joomla website, and you are experiencing slowdowns, then try disabling it. That should help tremendously. If it doesn’t, or if you are a bit hesitant about disabling this plugin, then please contact us. We’ll give you the right advice, we’ll fix the problem for you, we won’t charge you much, and you will gain top-notch programmers (modesty here, modesty…) and true friends for life!

Your Joomla Website Is Really Really Slow? Maybe It’s Bingbot!

A client of hours with a high traffic Joomla website called us and told us that everyday during peak hours, their website slowed to a crawl. So, as usual, we ssh’d to their server and we checked the slow query log and we didn’t notice anything unusual, which was unusual (even though the long-query-time was set to 1, which means that any query taking over 1 second [the minimum] was recorded into the slow query log). The reason why this was unusual was that the MySQL load was high, yet the MySQL slow query log was nearly empty.

So, and we have no idea why, we checked the Apache logs located under /home/domlogs (note that on some Linux servers, such logs are located under the /usr/local/apache/domlogs), and we tail’d the main daily (and current) access log using the following command:

tail -500 [ourclientjoomlawebsite.com]

and here’s a glimpse of what we saw:

157.55.39.224 – – [08/Jan/2016:09:11:10 -0500] “GET [relative-url-1] HTTP/1.1” 200 10105 “-” “Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)”
157.55.39.224 – – [08/Jan/2016:09:11:10 -0500] “GET [relative-url-2] HTTP/1.1” 200 9668 “-” “Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)”
157.55.39.224 – – [08/Jan/2016:09:11:10 -0500] “GET [relative-url-3] HTTP/1.1” 200 9491 “-” “Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)”
157.55.39.224 – – [08/Jan/2016:09:11:10 -0500] “GET [relative-url-4] HTTP/1.1” 200 9724 “-” “Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)”
157.55.39.173 – – [08/Jan/2016:09:11:10 -0500] “GET [relative-url-5] HTTP/1.1” 200 9188 “-” “Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)”

As you can see in the above, Bingbot (the search engine crawler released by Microsoft back in 2010 that serves their own search engine, Bing, as well as Yahoo) hit the website with 5 page requests in one second, and it was doing that every single second! Yes – we are not exaggerating. In fact, sometimes Bingbot would request more than 10 pages in one second, but the norm was to request 5 pages every second. How did we know that it was requesting an average of 5 pages every second? Well, by running the following shell command:

grep 'bing' [ourclientjoomlawebsite.com] | wc -l

The above command returned 56,348 when we ran it 3 hours after the logs rotated, which meant that 5.21 (56,348 / (3600 * 3)) pages were crawled by Bingbot every second.

Now we have seen many things in our lives, a three-headed dog (that was in Harry Potter and the Philosopher’s Stone, and the name of that dog was Fluffy), a three-headed monkey (that was in the Tales of Monkey Island game, although, and to be fair about it, we didn’t actually see the three-headed monkey, we were just told about it in the game, and so we assumed that it existed), a smiling taxi driver who drove smoothly and who conducted an excellent and fun conversation and who seemed to be happy about everything, and a warm Montreal winter that ended in early January (yes, that really happened back in 2009). But what we haven’t seen yet was a search engine robot trying to crawl a website with such aggressiveness and during peak traffic times – it wasn’t really normal. Obviously, Bingbot was the cause of this issue and we verified it by temporarily blocking it by adding the following lines to the .htaccess file (just after the RewriteEngine On):

RewriteCond %{HTTP_USER_AGENT} bingbot[NC]
RewriteRule .* - [R=403,L]

The above lines essentially blocked Bingbot from accessing (and thus crawling) the website, and after adding them we noticed that the load dropped to around 2 from around 10. So, the load issue was definitely caused by Bingbot. But, as you might have guessed, we can not keep this particular bot blocked since our client will lose his rankings with Bing.

So, we needed a way to tell Bingbot to slow down, and a quick research about the subject revealed Bing were aware about the load issue that their bot creates on the servers, and that they created a workaround to address it. The workaround was a small setting in the robots.txt file that tells Bingbot to take it easy a bit. In short, the setting will tell Bingbot how many seconds it has to wait until it hits the website with another request.

So, how to tell Bingbot to slow down?

You can tell Bingbot to slow down its crawl rate by adding the following to the end of your robots.txt file (note: we bolded and redded the last line since it is where the most important juice is, but you will need to add all the lines below, not just the red one):

# Slow down bing
User-agent: msnbot
Disallow: /administrator/
Disallow: /ads/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/
Crawl-delay: 5

The value of Crawl-delay (which is 5 in the above example), tells Bingbot to wait for 5 seconds between each page crawl.

We have to say that we were a bit (well, more than a bit) skeptical at first – we thought that Bingbot will just ignore that setting and keep crawling the website at their usual rate. But, we were pleasantly surprised when the next day, we noticed that Bingbot reduced its crawling speed to one page every 5 seconds, positively affecting the load on the server! The problem was solved!

But, what if Bingbot didn’t respect the “Crawl-delay” setting?

Well, in that case, we would have resorted to drastic measures. Such measures would have ranged from creating a special, module-less template for Bingbot to completely blocking this crawler by re-adding the above .htaccess lines. Of course, we would have to inform the client, who will have to choose between losing 50% of their traffic during peak hours because of Bingbot or losing about 10% of their traffic (the 10% of their traffic represents the incoming human traffic from Bing or Yahoo).

But wouldn’t setting the “Crawl-delay” setting to a high number result in an SEO hit specific to Bing/Yahoo?

Well, that’s a tricky question. Bing officially recommends keeping that Crawl-delay number low in order to ensure that Bing always has a fresh version of your website (it doesn’t say anything about SEO impact when you use that value). But we think that Bing‘s expectations of servers are somehow unrealistic – since not setting the Crawl-rate value will unleash Bing at turbo crawl (yes, you just witnessed the first oxymoron in this post) speed on your website, quickly swamping the server in case the website has many pages. While we’re not sure about the SEO implications with Bing, we think it’s of little importance when compared to the server health and responsiveness since the latter is a well-known SEO measure for Google. And, in all fairness, any website administrator will choose to accommodate Google over Bing any day!

So, if your website is very slow during peak hours and not-so-fast during non-peak-hours, then check your Apache logs for aggressive Bingbot crawling. If that’s the case, then try reducing the Crawl-speed value as per the above suggestion. If that didn’t work, or if the issue has nothing to do with Bingbot, then please contact us. We are here for you, we work hard, and we will solve any Joomla problem for a super affordable fee!

“DateTime::construct(): Failed to parse time string (Never) at position 0 (N)” Error on Joomla

We were busy most of the month doing emergency updates for our clients since Joomla had several critical updates in a row. Most updates went smoothly, but several didn’t. One update, for example, caused the following error to display on the homepage:

DateTime::__construct(): Failed to parse time string (Never) at position 0 (N): The timezone could not be found in the database

We tracked down the problem and we discovered that it had do with a change in a core file called calendar.php located under the libraries/joomla/form/fields folder. That change was incompatible with several 3rd party extensions (including the Mosets Tree extension, which was used by this particular website).

So, in order to fix the problem, we had to modify the calendar.php file the following way:

Just before the following line:

if (strtoupper($this->value) == 'NOW')

We added the following code:

if (empty($this->value) || $this->value == 'Never')
	$this->value = 'now';

And that fixed the problem!

But isn’t it a better to fix the problem in the affected 3rd party extension(s)?

Yes – it is better to do so. But we wanted to offer a generic solution in this post that will fix the problem on all extensions – instead of writing on how to fix the problem in every affected extension (and most likely we won’t cover all the affected extensions). And yes – we know – this is a core modification, but it is a quick and efficient modification (or at least we think it is!).

If you have the same problem on your website, then try modifying the calendar.php file as described above. If you want to fix the problem at the extension level, then please contact us. We’ll do it for you in not time and for a very reasonable fee!

Yet Another “The Connection Was Reset”/”No Data Received” Error on Joomla

A new client of ours emailed us and told us that whenever he visits his website, he’s seeing the following error:

The connection was reset.

The above error is displayed when he’s using FireFox. When he uses Chrome, his website also crashes but with the following (different) error:

No data received

Now, we worked on a similar problem before, but the difference is that on this instance, the problem was happening on every page on the frontend and the backend, while previously the problem was only happening in the backend when installing an extension.

So, the first thing that we did was that we disabled, in the configuration.php, gzip compression and FTP (we also removed all the FTP settings), but still, the problem was there.

We then disabled all the plugins and all the modules, and still we we were seeing the same error. That was very odd and we were perplexed – we just didn’t know where to look to find the cause of the problem (since what we were seeing was not a Joomla error, but a browser error).

Luckily, at around the same time the client started experiencing the problem, he received an automatic email from his server warning him that there was some corruption at the database level. That email (which he forwarded to us) unveiled the whole mystery.

You see, the automated email stated that the #__session table has crashed and that the system didn’t try to automatically repair it. So, we logged in to phpMyAdmin, we selected the database powering the Joomla website, and then we clicked on the #__session table, and we saw this message:

Table ‘./[database]/#__session’ is marked as crashed and should be repaired

So, we repaired the table using the following query (of course, we replaced #__ with the database prefix):

REPAIR TABLE `session`

And guess what? That fixed the problem!

But why did a corrupt session table cause this problem instead of just displaying a fatal error?

Honestly, we didn’t investigate the issue very hard in order to know why – but we’re sure that there’s a logical explanation. For now, you’ll just have to trust us that a corrupt session table will cause The Connection Was Reset/No Data Received error.

So, if you have the same problem on your website, try repairing the session table in phpMyAdmin. If that doesn’t work, then try repairing all the tables. If that still doesn’t work, then why not contact us? We’ll solve the problem quickly, cleanly, and for a very reasonable fee!

Your Joomla Website Is Slow After Changing Your Linking Structure? Read this!

Note: This post is slightly oriented for programmers. We just wanted to let you know before reading it in case you found it a bit too technical for your taste!

Another note: The solution presented in this post requires a core modification since it is addressing a bug in Joomla’s core. Proceed with caution and at your own risk, while keeping in mind that core modifications may be wiped out with a Joomla update.

A client of ours with a high traffic Joomla website (the website is a magazine), called us yesterday and told us that her website was experiencing some serious load issues. So we checked the server hosting the website and we noticed that the load was constantly in the double digits. This was odd because this issue happened all of a sudden, and they didn’t experience any spike in traffic. In fact, the traffic was relatively low this time of year for them (which is the case for many online businesses).

The first thing that we did was checking the slow query log, and it was full of the following queries:

# Time: 151228 16:39:29
# User@Host: db_user @ localhost []
# Query_time: 1.030854 Lock_time: 0.000249 Rows_sent: 11676 Rows_examined: 58413
SET timestamp=1449873569;
SELECT a.id, a.title, a.alias, a.title_alias, a.introtext, a.language, a.checked_out, a.checked_out_time, a.catid, a.created, a.created_by, a.created_by_alias, CASE WHEN a.modified = 0 THEN a.created ELSE a.modified END as modified, a.modified_by, CASE WHEN a.publish_up = 0 THEN a.created ELSE a.publish_up END as publish_up,a.publish_down, a.images, a.urls, a.attribs, a.metadata, a.metakey, a.metadesc, a.access, a.hits, a.xreference, a.featured, LENGTH(a.fulltext) AS readmore,a.state AS state,c.title AS category_title, c.path AS category_route, c.access AS category_access, c.alias AS category_alias,CASE WHEN a.created_by_alias > ' ' THEN a.created_by_alias ELSE ua.name END AS author,ua.email AS author_email,parent.title as parent_title, parent.id as parent_id, parent.path as parent_route, parent.alias as parent_alias,c.published, c.published AS parents_published
FROM #__content AS a
LEFT JOIN #__content_frontpage AS fp ON fp.content_id = a.id
LEFT JOIN #__categories AS c ON c.id = a.catid
LEFT JOIN #__users AS ua ON ua.id = a.created_by
LEFT JOIN #__categories as parent ON parent.id = c.parent_id
WHERE a.access IN (1,1) AND c.access IN (1,1) AND a.state = 1 AND (a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2015-12-11 22:39:28') AND (a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2015-12-11 22:39:28')
ORDER BY CASE WHEN a.publish_up = 0 THEN a.created ELSE a.publish_up END DESC, a.created;

Even though the query above was only taking a second, it was a problem, because there were many identical queries being executed every second.

Looking closer at the following line…

# Query_time: 1.030854 Lock_time: 0.000249 Rows_sent: 11676 Rows_examined: 58413

…we noticed that the rows being sent by the database to the (Joomla) application were 11,676 rows! Really? Why and where would Joomla need 11,676 rows from the #__content table in one shot? We thought it might be a very badly written module, and so we just emptied the index.php file located under the templates/[joomla-template] folder, and then we checked the load, which dropped a bit, but remained steadily in the double digits. This meant that the issue wasn’t caused by a module.

We then disabled all the published plugins, but, as we expected, it wasn’t the problem. We even switched to a basic template, but with no avail: the load was still very high!

So it wasn’t a module, it wasn’t a plugin, and it wasn’t the template causing the problem. What was it?

We then started debugging the problem, and we discovered that all of a sudden, the website started experiencing an abnormally high number of 404s. When we focused more on this issue, we realized that the high server load was caused by those 404 pages, and it didn’t take us long to know that the root of the issue was a bug in Joomla’s core.

You see, for a mysterious reason, when a user hits a 404 page, Joomla tries to load all the active articles from the database, and, when the database is very large, this can cause serious load issues on the server.

But is there a reason why Joomla does that?

No there isn’t. In fact, Joomla thinks that it’s not doing that. Let us explain by examining some code in the function getItems which is located in the category.php file (which is in turn located under the components/com_content/models folder).

if ($limit >= 0)
{
	$this->_articles = $model->getItems();

	if ($this->_articles === false)
	{
		$this->setError($model->getError());
	}
}
else
{
	$this->_articles = array();
}

As you can see, Joomla checks if $limit is greater or equal to zero, and if it is, it gets the articles from the database, if $limit is less than zero, then Joomla will not get anything form the database. In case you’re wondering, $limit tells Joomla to limit the number of articles retrieved from the database to its value, for example, if $limit is 20, then Joomla only asks MySQL to send 20 articles from the database (instead of 11k rows).

So, for the untrained eye, the above condition is solid: if $limit is less than zero, then do not get anything from the database. But what if $limit is NULL?

Well, if a value is NULL, then PHP will cast it (casting, in programming terms, means transforming a variable from one type to another) to a zero, and then Joomla will get all the active articles from the database (because technically, there is no limit imposed on the number of articles to be retrieved).

If you haven’t guessed it already, 404 pages on the Joomla website had a $limit of NULL, and thus each 404 page was technically loading 11k articles from the database into the memory – both creating a load on the database server and quickly exhausting the available memory.

In short, the condition if ($limit >= 0) is wrong – because if $limit is zero or equivalent to zero, then we will have a serious problem.

So, how did we fix the problem?

We fixed the problem by replacing, in the aforementioned category.php the following line:

$limit = $this->getState('list.limit');

with this one:

$limit = $this->getState('list.limit');
if (!isset($limit) || empty($limit))
	return array();

This ensured that we just return an empty array (as we should) whenever we have a $limit that is not even set, is NULL, or is set to zero.

But why did our client have this sudden spike in 404s?

Our initial guess at that was a change in the linking structure, and, after asking the client, we were proven correct in our assumption: the client made a massive change in the linking structure without making the necessary .htaccess redirection from the old links to the new links.

Now, if you, our dear, dear reader, are having slowdown issues after changing the linking structure on your Joomla website, then try the above solution and add the appropriate redirects in your .htaccess file. If you need help doing that, then please let us know. We are always available, our work is super professional, and we don’t charge much!

Joomla Website Slow After Updating Apache from 2.2 to 2.4

A client of ours, running a Joomla 3.4.6 website and not wanting to update to the seemingly rushed 3.4.7 asked us to update the PHP instance powering his website from PHP 5.4.44 (which has a known exploit when it comes to the serialization and unserialization of data) to close all known exploits on his server. As usual, we happily obliged.

We used EasyApache (from WHM‘s interface) to update from Apache 2.2/PHP 5.4.44 to Apache 2.4/PHP 5.5. Everything went smoothly, but, when we checked the website, it was extremely slow. It was taking literally 7-10 seconds to load each page, despite the fact that the website was using the System – Cache Joomla plugin for caching. The same website, before the update, was loading instantly.

The load on the server was very low (in the range of half a percent), and there were virtually no slow queries in the MySQL slow query log.

We remembered that EasyApache offered us the option to install XCache 3.2.0 for PHP, but we didn’t select that option. So we rebuilt Apache/PHP with that option selected (thinking that it would solve the problem), but the issue remained there after the update.

And then it hit us: Apache can be the source of slowness on a Joomla website (as we previously experienced), and so we checked Apache’s global configuration settings (by going, in WHM, to the Apache Configuration page and then clicking on the Global Configuration link), and we noticed that both the Server Limit and the Max Request Workers were set to very low, single-digit values, while the former’s default is 256, and the latter’s default is 150, and so we set both to their default values, we rebuilt the Apache configuration, and then we restarted Apache. Unsurprisingly, that solved the problem!

But why did EasyApache set these configuration settings to very low values?

We really have no idea – but we don’t think it was a glitch as we used the highly stable EasyApache 3. We think that these settings were intentional, thinking that they would make the website more secure and more resilient to DoS (Denial of Service) attacks. Ironically, by trying to protect the website against DoS attacks, EasyApache created the same effect that a DoS might have on a website.

If you’re having any performance issues after (fully or partially) updating your LAMP environment, then check Apache‘s settings, as the problem might be there. If you have already checked and you have found nothing outside the ordinary, then go ahead and contact us. We will find a solution, we will implement the solution, we will fix the problem, and you won’t be charged much!

“Fatal error: Call to undefined function mb_regex_encoding()” Error in Joomla

Note: The guide described in this post, although straightforward, is probably better performed by a system administrator as there’s always a chance of a glitch while tinkering with the server’s Apache and PHP installations.

Last week, we were commissioned to create a microsite for a large clients of ours. The microsite consisted of copying a partner’s website into their website (they recently bought that partner). We were excited because creating microsites is always a fun and straightforward job.

We usually use K2 for creating microsites (since it has the very powerful and adaptable “extra items“), and this time was no different… But (you knew there was a “but” somewhere, didn’t you?), after installing K2 and going to the K2 extension in the backend, we encountered the following error:

Fatal error: Call to undefined function mb_regex_encoding() in /home/[user]/public_html/components/com_k2/helpers/route.php on line 94

Luckily, we did encounter this issue before (but with a different extension), and so we knew exactly what the problem was… In short, it was a missing module (a PHP module, and not a Joomla module) that should have been bundled with the PHP instance installed on the server, but wasn’t. So, in order to fix the problem, all that needed to be done was to re-install PHP with that missing module (called Mbregex, short for Multibyte Regular Expression). Here’s how to do that:

  • Login to the WHM interface hosting that website.
  • Search for the word “EasyApache 3” in the search textbox on the top left and then click on its link.

  • In the selected configuration (the configuration with the radio button checked next to it, under Previously Saved Config), click on the settings wheel (just under Actions).

  • Click on Next Step when you are in Apache Version and PHP Version (do not change anything). Once you are in the Short Options List page, click on Exhaustive Options List (at the very bottom).

  • On the Exhaustive Options List page, click on the checkbox next to Mbregex, and then click on Save and Build at the bottom of the page. You will be asked this question “Recompile Apache and PHP now?” to which you should answer “OK” and then “I understand”.

  • Now wait and do not close your browser or the current page! As a reward for your 10-minute patience, a popup will appear where you just have to click on Save New Configuration (scroll down to the end of the text in the popup in order to see the Save New Configuration button). Once you click on it, the new Apache/PHP configuration settings will be applied and the problem will be resolved!

That was easy, huh? And the best thing is that the downtime is literally less than a minute to resolve this problem (you will have an unavoidable downtime since Apache, the web server, needs to be restarted during the process). Note that your website will not be down during the installation process (again, which takes around 10 minutes).

But what if the website is not on a WHM based server or is on a shared hosting platform?

If your website is on a Plesk powered server, then doing the above is not as straightforward and can get slightly complex. We suggest you ask us (or a system administrator, or your hosting company) to do that for you.

If you are on a shared hosting, then you are at the mercy of your host, or may or may not be willing to do it for you (since installing Mbregex on your account means installing it on the whole server).

If, even after following the above guide you are still seeing the same error, then try clearing your Joomla cache and see if it helps. If it doesn’t, then it’s probably time to contact the Joomla experts (that’s us!) who will solve the problem for you in not time and for a really, really affordable fee!

Blank Page on the Article Manager in Joomla’s Backend – How to Resolve

This very morning, the head administrator of a Joomla intranet website belonging to an extremely reputable university in the US Northeast (that university is a client of ours), called us and told us that whenever he goes to the Article Manager page on Joomla’s backend, he sees a blank page (despite the fact that his Error Reporting was set to Maximum in the Joomla’s global configuration). So we told him the following, just close your browser (all instances of the browser), and then open it again, and then try visiting the Article Manager page.

So he did that, and, unsurprisingly, we heard a loud “Huh?” on the phone immediately followed by “You guys are geniuses! It’s working now! But how?”

While we were tempted to claim that we performed some magic on our end to solve this problem, our open-knowledge nature eventually prevailed and we explained to the customer why closing the browser and re-opening it solved the problem. Now let us explain it for you, our dear reader!

Over the years, we have worked on Joomla every single day, and so we know almost everything about it, and we know the cause of the absolute majority of snags that Joomla administrators hit. You see, our client, when he was in the Article Manager page, he filtered on a specific category, and then, on the top right, he selected All instead of 20 to display all articles belonging to that category. After he finished his work on that particular category, he removed the category filtering and the website crashed because Joomla was trying to load all the articles in all the categories (since All was still selected), which made his PHP instance run out of memory.

But, is there a way to solve this problem without closing the browser?

We understand that some times you are running some important applications on the browser and you just can’t close it, and so another way to solve the problem would be to use another browser. But, if you don’t have another browser (some corporations restrict their employees to using one browser, which is typically Internet Explorer), then you can just go to phpMyAdmin, select the database powering your Joomla website, and then clear the #__session table by issuing the following:

TRUNCATE TABLE #__session

But, what if I really want to see all the articles in one shot?

If you have that very strange urge, then you can just increase the memory available for PHP (we have explained how to do this before here). Keep in mind that increasing the memory is a two-edged sword, as while it will fix your problem, it may lead to the whole system running out of memory and thus resorting to swap which will slow down the whole server, eventually crashing the website.

We hope that the above guide helped you fix your problem. If you still have the same issue even after closing your browser and increasing the memory, or if you feel that this issue is a bit too technical for your taste, then please contact us. We’ll do the work for you professionally, quickly, and for a very competitive fee!

Monitoring the Logs: A Proactive Technique to Ensure Your Joomla Website Remains Secure

OK – we have to admit it. We’re paranoid and we’re afraid of our own shadows, we’re also afraid of clowns (they really are scary), adult strangers talking to us in the streets, and, of course, being sucked down the bathtub drain (baths are always a nightmare). But our greatest fear is seeing a client’s website hacked, and that fear is so powerful that it forces us to be extremely proactive when it comes to website security.

Of course, proactiveness (or should it be proactivity for those grammaticians out there?) when it comes to website security manifests itself in many different ways at itoctopus, but one of them is monitoring the logs – specifically the Apache logs.

Logs, for those who don’t know (or for those who try to avoid looking at them) provide a wealth of very precious information about a website when it comes to security. For example:

  • Who’s logging to the website’s backend?

    One of our major clients is located in Chicago, IL. So, we examine every IP that logs in to backend of the website, and if we see an IP that is using the backend from any area outside Chicago, then we scrutinize all the activities performed by that IP and we tell our client to make sure that the login was (or was not) a false positive.

  • Which PHP files are being used?

    By default, the only PHP file accessed directly by Joomla (Apache) should be the index.php file, so, if, we find in the logs that Apache is successfully accessing another PHP file, then we examine that file very thoroughly to make sure that it’s a legitimate file. If it is not, the we get the file creation data, we delete it, and then based on the creation date of file we search through the logs in order to have an idea how it got there in the first place.

  • Which IPs are draining the website’s traffic?

    Examining the Apache logs allows us to find which IPs are using up the majority of the website’s bandwidth. Once we know those IPs, we can then choose to either ignore them (if they are legitimate crawlers/visitors) or block them (if they are visiting the website for malicious reasons).

  • Which pages/file are “404s”?

    Google Webmaster Tools provides some in-depth information about your 404 pages – but it’s still an external tool, and it doesn’t tell you much about the 404 files (such as 404 CSS files, 404 JavaScript files, etc…). Examining the logs allows us reveal which pages/files are being accessed extensively by the visitors but do not exist. Of course, you might be wondering, “what does that have to do with security?”. Well, a lot, since many attackers scan the website for vulnerable files randomly – so this allows us to catch the attackers’ IPs and block them. Also, performance-wise, addressing 404s by either blocking the request to the page/file altogether or by redirecting to another page will lessen the load on the both the web server and the database server.

If you want to have a robust and secure Joomla website, then monitoring/examining the Apache logs can help you tremendously (this should be a full time job by the way on large websites). If you need some guidance or if you need us to do this job regularly for you, then please contact us and we’ll be certainly happy to do so. Note that our super duper affordable fees apply!

Optimizing JComments in Joomla’s Backend

A client of us called us and told us that they were were experiencing huge slowdowns when using JComments in the backend – such slowdowns were affecting the whole website and sometimes lasting for several minutes! After a quick investigation in the slow query log, we were able to isolate the 3 queries causing the problem:

  • Query 1: (sometimes taking about 60 seconds to execute):

    SELECT jc.*, (SELECT COUNT(*) FROM `#__jcomments_reports` AS r WHERE r.commentid = jc.id) AS reports,jo.title as object_title, jo.link as object_link,u.name AS editor
    FROM `#__jcomments` AS jc
    LEFT JOIN `#__jcomments_objects` AS jo ON jo.object_id = jc.object_id AND jo.object_group = jc.object_group AND jo.lang = jc.lang
    LEFT JOIN `#__users` AS u ON u.id = jc.checked_out
    ORDER BY jc.date desc LIMIT 0, 20;

  • Query 2: (sometimes taking about 5 seconds to execute):

    SELECT DISTINCT(lang) AS name
    FROM #__jcomments
    ORDER BY object_group ASC;

  • Query 3: (sometimes taking about 5 seconds to execute):

    SELECT DISTINCT(object_group) AS name
    FROM #__jcomments
    ORDER BY object_group ASC;

After isolating the queries, our next step was to locate the file containing these queries, and it didn’t take us long: it was the file comments.php located under the administrator/components/com_jcomments/models/ folder. So, all we had to do was to optimize the queries in that file. This was done the following way:

  • At the very beginning of the function getListQuery, we added the following code:

    $sql = 'SELECT COUNT(*) FROM ' . $this->_db->quoteName('#__jcomments_reports') . ' AS r WHERE r.commentid = jc.id';
    $this->_db->setQuery($sql);
    $numReports = $this->_db->loadResult();

    We then changed the following line:

    $reportsSubQuery = ', (SELECT COUNT(*) FROM ' . $this->_db->quoteName('#__jcomments_reports') . ' AS r WHERE r.commentid = jc.id) AS reports';

    to:

    $reportsSubQuery = ', ' . $numReports . ' AS reports';

    This way we split the very heavy query into two queries which dramatically improved the performance (subqueries should always be avoided since they create a huge load on the database server), and by dramatically we mean that the execution time was reduced from 30-60 seconds to a very tiny fraction of a second.

  • At the very beginning of the function getFilterLanguages, we added the following code:

    $objLanguage= new stdClass();
    $objLanguage->name = 'en-GB';
    return (array($objLanguage));

    The 3 lines above ensured that JComments didn’t have to scan all the comments to know the available languages. Since we know that our client’s website only uses English, all we needed to do was to hardcode the returned array to only contain the English language (hence the en-GB in the code). If you’re using Spanish, for example, then you should replace en-GB (in the above code) to es-ES. Note that if you have comments in multiple languages, then you should create an object for each language (e.g. $objLanguage1, $objLanguage2, etc…) and then return an array containing these objects (e.g. return array($objLanguage1, $objLanguage2, …)).

  • At the very beginning of the function getFilterObjectGroups, we added the following code:

    $objObjectGroup1= new stdClass();
    $objObjectGroup1->name = 'com_content';
    $objObjectGroup2= new stdClass();
    $objObjectGroup2->name = 'com_k2';
    return (array($objObjectGroup1, $objObjectGroup2));

    The above code will work for websites using Joomla’ core and/or K2 for content management (the website we were working on was using both). If you are using another extension for content management in conjunction with JComments, then you should add it to the code above (while, at the same time, removing the unnecessary ones, such as com_k2 if you’re not using K2).

After implementing the above code changes, our client experienced huge performance gains and the MySQL server was relieved from doing unnecessary work.

Now, in case your website is suffering from slowdowns when you’re working with JComments in the backend, then try implementing the above optimizations. If you are still having problems (even after doing the above), or if you just don’t want to touch any code, then please contact us. We will definitely help and we’re confident that you will see huge improvements on your website in very little time and for a very little cost.

A Super Fast Way to Clean a Hacked Joomla Website

Note: This post assumes you have cPanel access to your website. The instructions for other hosting interfaces are different and are not covered in this post.

Warning: The method described in this post will wipe out any core modifications that you might have on your Joomla website. You have been warned!

Another warning: As explained in this post, this method may or may not work on your website (depending on the type of hack) – so there are no guarantees whatsoever that your website will be “clean” once you follow the below.

For years now, we’ve been getting so many tasks to unhack Joomla websites every month. What’s interesting is that these tasks never bore us, because quite a few of them are unique, and by unique we mean that the exploited websites suffered from hacking techniques that we have never seen (or fixed before). But about 50% (or maybe even more) of these tasks are standard (a standard Joomla hack, as we see it, is where core, and only core, Joomla files are hacked – anything other than that is a non-standard hack), and this has helped us devise a super fast method to cleanup a standard-hacked Joomla website.

You might be wondering – didn’t we write about this before? Well, not really, because the method described in the previous post is fast, but this method is super fast. On the flip side, the method described in this post is a hit or miss: it might work on your website or it might not work, while the other method is more of a solid process to cleanup Joomla websites.

So, without further delay (we know that you’re probably anxious and don’t have time to read literature), here’s our super fast method that you can use to clean your Joomla website from malware:

  • Do not visit your website (backend or frontend).
  • Backup your website. Yes, we know it’s bloated with malware, but you should back it up!

  • Ensure that index.php is the only PHP file that Apache can run as described here.

  • Login to the cPanel account hosting your Joomla website.
  • Click on File Manager, and then browse to the libraries/cms/version folder located under your Joomla website.

  • Open the file version.php and take note of these 2 variables: $RELEASE and $DEV_LEVEL. These variables (a variable, in the programming world, is… Well, we don’t know how to explain it in layman terms, but, if you don’t get it, that’s OK…) will tell you the exact version of your Joomla website. For example, if you have something like…

    public $RELEASE = '3.4';
    public $DEV_LEVEL = '5';

    …then your Joomla version is 3.4.5.

  • Download the exact version of your Joomla website from Joomla’s official website (you can always google your Joomla version and you will find the download page for your version somewhere in the first 3 links). Note that you should download the full release, and not the update.

  • Upload the file that you have just downloaded (from joomla.org) through cPanel’s File Manager to the main directory of your Joomla website – e.g. the /home/[username]/public_html directory – and extract it there (just right click on the file, and then click Extract).

  • Once the file is extracted (unzipped), then remove the installation folder and the Joomla zipped file that you have just extracted.

  • Check if your website is now clean!

If this method works for you, then congratulations! You’re one of the lucky ones! We do suggest, however, that you investigate how your website got hacked in the first place to prevent the same thing from happening again. If it doesn’t work, then don’t worry, we’re here for you! Just contact us and we’ll clean your website in no time and for a very affordable price!

“Files – Search”: A Free Joomla Plugin that Will Search Through Files

Ah search plugins, we love them, we truly do! Every time we get a job to create a search plugin we fight amongst ourselves as to who will create this plugin! We have created search plugins that will search core Joomla tables, non-core Joomla tables, specific modules, etc… We even created search plugins that will search data located on a remote server (through a web service). However, there is one very special search plugin that we have developed and refined over the years, we like to call it the “Files – Search” plugin (not a very original name, huh?). What this search plugin does is slightly different than what other search plugins do: instead of searching a table in the database for a keyword, it searches files in a specific directory for that keyword!

Since we are very generous, we have decided to release the plugin for our loyal readers for free: you can find it here.

What does this plugin do exactly?

The “Files – Search” plugin will search through all the files in the “Search Directory” (which is a setting in the plugin) and will return the URL of each file that has the searched text in its content along with an excerpt (as well as the creation date of the file). The plugin will then send its search results (e.g. the matching files) to the search component, which will then merge them with the search results of other search plugins, and will eventually display them on the “Search Results” page.

What kind of filetypes can this plugin search?

Currently, the plugin can search Text files (.txt, .text), HTML files (.htm, .html), PDF files (.pdf), MS Word documents (.doc, .docx), MS Excel documents (.xlsx), and MS PowerPoint documents (.pptx).

Is this plugin stable?

Yes – we can safely say it’s very stable. In fact, we use it on quite a few large production websites and it performs amazingly. So, yes, it’s really stable!

Is installing, configuring, and operating this plugin a hard task?

No – it’s really quite easy. In fact, all you need to do after downloading the plugin from our website and installing it on your Joomla website is to modify the only setting the plugin has which is the “Search Directory” (to make it point to the folder containing the documents that you would like to search for). Afterwards, you just have to enable the plugin and that’s it!

Are there any limitations to this plugin?

Currently, the plugin will only search one folder (recursively) – so, if you want to search multiple folders, then please contact us and we’ll give you a quote for that (note that our fees are very competitive). Additionally, the plugin will only search files of specific types (these types are listed above) – again, if you want to search for other file types, then you will need to contact us.

Will the plugin work on all versions of Joomla?

The last time we have used this plugin was very recently (on a Joomla 3.4.5 website), but we’re confident that it’ll work flawlessly on Joomla 2.5 websites. As for Joomla 1.5 (if you still haven’t upgraded yet), then the plugin might work without any modification (but we suspect that), and if it doesn’t work then please contact us, we’re sure we can make it work for you in very little time.

Will the plugin search PDF files?

Yes (we mentioned this above but we wanted to assert this point), and what’s interesting is that it does that without installing any 3rd party library on the server hosting your Joomla website. It can also search the majority of Microsoft Office documents (also without installing any 3rd party library on the server).

So, is this plugin free?

Yes – it’s free: you can modify it whichever way you like and use it to fit your needs. We’re also against the practice of linking back to our website from our plugins, so you can rest assured that the plugin is free and clean.

However, keep in mind 2 things:

  1. We don’t provide free support for any of our free plugins. If you want support then please check our fees!
  2. Although we have made sure that this plugin is safe and will not cause any issues, please keep in mind that you are using this plugin at your own risk. We cannot be held liable for any issue directly or indirectly caused by the use of this plugin. This free plugin is provided as-is and comes with no warranty whatsoever!

We hope you will enjoy using this plugin the same way we have enjoyed building it. Should you have any questions about it or should you want us to modify it for you in any way, then don’t be a stranger; all you need to do is to contact us (again, keep in mind that our super affordable fees apply)!

The Quickest and Best Way to Update a Joomla Website

Note: This post is about updating a Joomla website, and not migrating it. See the difference between the two here.

A few weeks ago, Joomla released Joomla 3.4.5, which is a critical security update that closes a SQL injection exploit existing in previous versions of Joomla. At the time of its release, we got so many calls from Joomla administrators who had issues updating their Joomla website. This is normal and we did help them address the issues that they faced. However, we also got a not-so-small number of calls from Joomla administrators who were intimidated from the whole “update” concept. They felt that 1) they will run into issues, and 2) the update will take a lot of time, and so they asked us about the fastest and best way to update a Joomla website. We told them that they could follow our easy guide of updating Joomla website. Naturally, they asked us about it, and we told them that we’ll publish it soon on our website! And here it is, our super-duper easy and quick guide for updating Joomla websites (note that the following guide applies to a WHM/cPanel environment):

  • Backup your Joomla website. Make sure you backup both the database and the filesystem. Do not proceed without backing up your website. You have been warned!
  • Make sure that your website’s backup works. Sometimes, backups can be corrupt, and you can’t really restore a website with a corrupt backup!

  • Login to the server hosting the Jooma website through ssh (we usually ssh to a server using putty).

  • Go to this folder: /home/[cpanel-username/public_html using the cd command. E.g.:

    cd /home/[cpanel-username]/public_html

  • Download the latest version of Joomla to the server. This can be done using the following command:

    wget https://github.com/joomla/joomla-cms/releases/download/3.4.5/Joomla_3.4.5-Stable-Full_Package.zip

    Note that the above link must be replaced with the latest Joomla download link, which should be copied from the joomla.org/download.html page (it should be the Full Package link).

  • Add the following line to the beginning of the .htaccess file (located under the root directory of your Joomla website):

    deny from all

    The above ensures that your website is not available during the update process – otherwise your visitors will be redirected to the installation folder once you do the next step.

  • Extract the downloaded file onto your Joomla website. You can do this using the following command:

    unzip Joomla_3.4.5-Stable-Full_Package.zip

    (Note: Answer “A” to update all files when you’re prompted by the system.)

  • Remove the installation folder by issuing the following command:

    rm -Rf installation

  • Remove the downloaded Joomla install by issuing the following command:

    rm -Rf Joomla_3.4.5-Stable-Full_Package.zip

  • If you’ve ssh’d as root, then you will need to issue the following commands to correct the file ownership on the Joomla files:

    chown -R [cpanel-username] *
    chgrp -R [cpanel-username] *

  • Open the .htaccess file and remove the line that you have added earlier in the process.

  • That’s it. Your website is now fully updated.

Note that if you have modified some core files on your Joomla website, then you will need to backup these files before starting the process, and then you will need to restore them after the above process is done. If you don’t do that, you will lose all the core modifications done on your website.

But why do the above when you can just update your website by clicking on a button?

Well, because that button click doesn’t work smoothly all the time. Additionally, by extracting the Joomla install over the current website, we avoid any database updates to the website (database updates can cause compatibility issues), since we’re just overwriting files.

If you have problems updating your Joomla website, then try our guide, it should help tremendously. If it doesn’t work for you or if you think that it’s not the easiest guide to follow, then please contact us. We’ll update your website in no time and for a very affordable fee!

“Bad Request: Your browser sent a request that this server could not understand.” Error When Trying to Access Your Joomla Website

A client of ours emailed us a few days ago and told us that whenever he tries to access his website, he’s seeing the following error:

Bad Request: Your browser sent a request that this server could not understand. Size of a request header field exceeds server limit.

He told us that he was the only one in his company seeing this problem, but that he had no idea whether any of his website’s visitors was having the same problem. He asked us to investigate, and so we did.

A quick research revealed that the issue was related to his browser having a huge cookie that was bigger than the maximum size allowed by the server. So, a fast solution to the problem was to ask him to clear his cookies on his browser. However, that didn’t technically solve the problem, it just made the problem go away. But, we know that these problems are like seagulls, when you shoo them away, they will leave, but they will eventually return.

So, we had to solve this problem at its root, and solving it at its root meant that we had to increase the maximum size of the cookie allowed by the server at the server level. We did this the following way :

  • We ssh’d to the server as root.
  • We opened the file httpd.conf located under the /usr/local/apache/conf/ folder.

  • We added the following line to the very beginning of the file:

    LimitRequestFieldSize 16384

  • We saved the file and we restarted Apache.

  • The problem was solved!

Note that the location of the httpd.conf file may vary from one Linux distribution to another. The location specified in the guide above applies to a CentOS (WHM) server.

Additionally, note that the above solution can only be implemented if you have root access to your server (e.g. if you have a VPS or a dedicated server). If you are on a shared hosting, then your best option would be to contact your host and ask them to increase the LimitRequestFieldSize value for you (keep in mind that your host will probably refuse as this will open a whole can of worms for them – so, your other best option would be to move to a VPS or a dedicated server).

But, what is the default value of LimitRequestFieldSize?

The default value of LimitRequestFieldSize is 8190 (speaking a bit more technically, the 8190 is the default maximum size of the HTTP request header field). Typically, most websites do not need to go over the default limit, but some websites store a lot of data in their cookies, causing the limit to be exceeded, and eventually causing the problem described in this post.

Can the problem be fixed at the .htaccess level?

Unfortunately, no, it can’t be fixed there. It can only be fixed at the httpd.conf level (hence the problem cannot be fixed if you are on a shared hosting).

If you have the same problem on your website, then try, before clearing your cookies, to fix the problem at the server level by modifying the httpd.conf file. The reason why it’s better to fix the problem before clearing your cookies is to make sure that you actually fixed the problem. If the problem is still there, then please contact us. We’ll fix it for you in as little time as possible and for a super affordable fee!

Super Login: A Free Joomla Plugin that Allows a Super User to Login as Any User

At itoctopus, we test our clients’ Joomla websites constantly, so much that we have developed Joomla plugins that will help us in the testing. One of the plugins that we have developed and that we constantly use is the Super Login plugin, which allows us to login as any user to the Joomla website with that user’s username and our super user password.

For example, if we have jeff as one username on a certain Joomla website that we manage, and the password for jeff is jeff123, and there is a super user called admin on that same website with admin123 as password, then we can (if we have the Super Login authentication plugin installed and enabled) login as jeff with the following credentials:

Username: jeff
Password: admin123 (instead of jeff123)

What’s interesting is that once we are logged in as jeff using our password, we will have all the rights/privileges for the jeff username, it’s exactly like logging in as jeff (think about it as if jeff has one username and two passwords).

We have developed and maintained this plugin for years now but we have just revamped it for public release, and because we are very generous, we are offering it to our readers for free. It can be downloaded here.

The Super Login plugin is revamped for Joomla 3.x and is built on top of the Authentication core Joomla plugin. Joomla 2.5.x compatibility is possible with little or no modification – we used to use it on Joomla 2.5.x, but we have modified it heavily since (that’s why were not sure if it’ll work or not).

The plugin, for security reasons, will not allow a super user to login as another super user (if you really need to allow this then a quick modification to the code will address this limitation). Additionally, and also for security reasons, the plugin will only check 10 super users for a matching password. So, if you have more than 10 super users, then only the first 10 will be considered (in any case, we don’t think that any Joomla website should have more than 10 users).

The way the plugin works is as follows:

  • It first checks whether the username provided actually exists. If it doesn’t, then the plugin throws an error message and the authentication fails.
  • It then checks if the username belongs to a super user. If it does, then it throws an error message and the authentication fails.

  • It then gets the Group ID for super users (super administrators). Typically their Group Id is 8, but the plugin is intelligent enough to handle any change in that number on heavily modified websites (or in case Joomla decides to assign super users another Group Id).

  • The plugin then grabs a maximum of 10 super users from the #__users table for comparison.

  • It then loops through the super users, until it finds a super user with a matching password. Once it does, it logs the user in with the username entered.

So, is this plugin secure?

We can confirm that it’s very secure. Again, it’s built on top of the Joomla core authentication plugin, which is, undeniably, very secure. Neverthless, it’s important to note that the plugin doesn’t have any DoS protection, which is why we have limited the number of checked super users to 10.

When is this plugin useful?

This plugin is useful when you want, for example, to login to the backend as another user to see what the other user sees and what he can do (after creating that user or after changing the role granted to that user). It’s also useful in the frontend for the very same reasons.

Why is this plugin free?

Well, it’s obvious isn’t it? Because we love Joomla and we love our readers, and we’re doing out best to keep Joomla at the helm with useful, clean, and sophisticated extensions. And, before you ask, it’s really free, there are no hidden links anywhere and you don’t have to buy anything (unless, of course, you want us to support it).

The plugin’s installation is super simple. Just install it from the Extension Manager, and then enable it (make sure that it’s ordered last among enabled authentication plugins). If after installing it and enabling it, the plugin didn’t work for you for some reason, then please contact us, but note that we charge fees for supporting this plugin. Also, last but not least, please keep in mind that this plugin is provided as-is and comes with no guarantees whatsoever, and we can’t be held responsible for any issue, either major or minor, resulting directly or indirectly from the use of this plugin.

Administrator Menu Not Appearing in Joomla’s Backend

We had a very curious case today: an occasional customer of ours emailed us and told us that for the past few months they weren’t able to see the administrator menu in the backend. They temporarily addressed the problem by installing an extension called Admin Menu Manager, which provided an alternative for generating the admin menu.

Of course, that wasn’t an ideal solution, and that’s why they came to us. The first thing that we did was uninstalling the Admin Menu Manager extension, and when we did that, the backend was menuless (yes, we just invented a word!).

At first, we thought that there was an error somewhere in the menu (non-fatal error) that was causing this issue, so we set Error Reporting to Maximum in the global configuration settings, but the website was clean as snow, it was errorless (that’s two words already!), which really meant one of the following 5 things:

  1. There is corruption at the ACL (Access Control List) level.
  2. The Administrator Menu module is disabled.

  3. The Administrator Menu module is trashed/deleted.

  4. The Administrator Menu extension is disabled.

  5. The Administrator Menu extension is uninstalled.

So, the first thing that we checked was the ACL corruption, and, after a quick glimpse at the ACL tables in the database, we were able to confirm that the issue was not there.

The next thing to do was checking the Extensions -> Modules page (which we accessed using the following URL: [www.ourclientjoomlawebsite.com]administrator/index.php?option=com_modules ) to see if there was a module called Admin Menu of type Administrator Menu with a Special access and assigned to the menu position (of course, we needed to switch the Filter on the left from Site to Administrator first) – there wasn’t! Checking the trashed modules revealed nothing, which primarily meant that the module was trashed and then deleted. Weird…

So, we tried recreating the module by just clicking on the New button (on the top left) when in the Modules page, but, to our surprise, there was no option to create an Administrator Menu module, the list of possible modules started with Administrator Sub-Menu. Huh… This probably meant that the Administrator Menu extension was either disabled or uninstalled.

So, we went to the Extensions -> Manage page (which we accessed from [www.ourclientjoomlawebsite.com]/administrator/index.php?option=com_installer), and then we clicked the Manage tab on the left, and then we searched for the extension named Administrator Menu (we expected it to be disabled), but it wasn’t even there! This core extension was uninstalled by someone (or something). This meant that we had to re-install it, and so, we created an installable version of this core extension manually the following way:

  • We downloaded a fresh Joomla install.
  • We extracted that install and then we navigated to the administrator/modules/mod_menu folder.

  • We then copied the en-GB.mod_menu.ini and the en-GB.mod_menu.sys.ini files from the administrator/language/en-GB folder to the above directory (administrator/modules/mod_menu).

  • We then compressed (zipped) the contents of the mod_menu directory (we made sure that we compressed the contents of the directory, and not the directory itself). Note: the resulting compressed mod_menu can be found here (note that this is extracted from a Joomla 3.4.5 install, but it can be used on any Joomla 3 website).

  • We installed the menu extension created in the previous step by going here [www.ourclientjoomlawebsite.com]/administrator/index.php?option=com_installer&view=install and then uploading the zipped file, and voilà, the backend menu appeared in all of its glory! The problem was fixed, hooray!

If the Administrator Menu is not appearing in the backend of your Joomla website, then try following the above guide to locate and fix the issue. If it doesn’t work for you, or if you feel that it’s a bit over your head, then please contact us. We will fix the problem for you at a very affordable price and in the cleanest way possible!

20 Questions You Should Be Asking Yourself If Your Joomla Website Is Slow

A substantial chunk of our work consists of optimizing Joomla websites. As such, we have gathered over the years a vast experience in that particular field and we almost know every variation of the problem as well as its causes. In this post, we would like to share our knowledge with our readers by providing them with the 20 questions that we usually ask ourselves (well, they’re really 19 questions + a final question that only you – and not us – can ask) when we’re handed over a Joomla optimization task.

  1. Is your Joomla website really slow?

    Yes, that might be a weird question, but let us tell you a little story after which you will probably feel that this first question isn’t that weird after all! Yesterday, we had a client whose Joomla website was loading in 14 seconds (yes, fourteen seconds). After thorough investigation and a lot of optimization, we discovered that the main problem lied with a 3rd party JavaScript widget that is used to load ads (and no, it’s not DoubleClick). Obviously, this is not a Joomla problem… So, before (mis)judging your Joomla website, check whether you have some JavaScript code (ads, social widgets, etc…) that is causing the delay in the loading of the website.

    Google PageSpeed Insights is a great tool that tells you whether the problem is from your Joomla website or not. If your server response time is higher than 1 second, then the problem is definitely from your website, otherwise, it might be something else.

  2. Is your Joomla website slow only at the time when you’re posting articles to the website?

    If you notice a significant lag on your Joomla website when you’re posting new articles, then this is usually primarily related to one of the following: Smart Search and/or the assets table. Let’s start with Smart Search: every time you create/update an article on the website then the Smart Search plugin, if enabled, will insert all possible search queries in the Joomla database (in *finder* tables), an action that causes a huge load on large Joomla websites. The irony is, the absolute majority of Joomla websites don’t even use the Smart Search functionality but still, they have all the Smart Search plugins enabled. Obviously, addressing this problem consists of disabling all the plugins starting with Smart Search in Joomla’s backend, and, if you’re one of the rare Joomla administrators who actually use Smart Search, then we suggest you try Sphinx to power your Joomla’ search engine, which is much, much faster and way, way lighter on your server.

    Moving to the assets table, which was discussed before, then all you need to do is to regularly clean it (clean it, and not clear it!), and you shouldn’t have any problems because of it anymore.

    Of course, there are other issues that might be causing slowdowns on the website when new articles are posted, but the root of these issues is typically a plugin that runs when articles are saved (so check your 3rd party plugins one by one until you find the culprit).

  3. Is your Joomla website slow during off hours?

    A couple of months ago, a client called us and told us that every day, at around 2 AM, his website slowed down to a crawl for about 30 minutes. A quick investigation of the issue revealed that the website (which was a huge website) was being backed up at the time, and, during the database backup, MySQL was experiencing a very high load causing the website to be super slow (often crashing the website for a few minutes). Fixing the problem consisted of dropping all the test databases/test tables, clearing large tables which data is not needed, disabling database powered stats (such as disabling Exim’s stats), and excluding some directories from being backed up (of course, this has nothing to do with MySQL, but it lessens the load on the file system).

  4. Are you using InnoDB or MyISAM?

    There are many existential debates in this world, such as the chicken and the egg debate (which came first), the white bread vs. whole wheat debate, the computer vs. the tablet debate, and, of course, the MyISAM vs. the InnoDB debate. We’re among those who believe that MyISAM is significantly faster than InnoDB (MyISAM and InnoDB are both storage engines in MySQL), and we are basing our information on the many Joomla websites we have optimized. So, if you are using InnoDB, then try switching to MyISAM and see if that helps improve the speed on your website. (By the way, if you really want to know, we think that the chicken came before the egg, we think that the white bread is better and tastier than whole wheat bread, and we definitely prefer computer over tablets).

  5. Are you positively, absolutely sure your website is not hacked?

    Some Joomla websites that we were commissioned to optimize turned out to be hacked. These websites were grabbing remote content (using curl) from malicious websites, and this was causing the slowdown as these malicious websites were very slow to respond and the website had to grab the malicious content before loading. We examined a case here. Scan your website, or, better yet, hire some experts (such as, ahem, us) to check whether your website is hacked or not.

  6. Have you checked your firewall log?

    In one rare instance, the slowness of a Joomla website that we examined was caused by the firewall. In short, the firewall was over-protecting the website from flooding (with SYN_FLOOD), and was incorrectly interpreting multiple connections made from the same page (to load images, CSS, and JS files) as a DoS attack. Fixing this problem consisted of a quick call to the host which resolved the problem by tweaking the firewall settings.

  7. Have you checked that your Apache configuration can handle the number of connected clients that you’re getting?

    In Apache’s configuration, there is a setting that will cap the maximum number of clients (workers), and each client is needed to serve on visitor. If you get a number of simultaneous visitors higher than your specified number of clients, then Apache will queue those visitors, resulting in a dramatic overall drop in the performance of the website. To check whether you have this problem or not, you will need to check the Server Status -> Apache Status page in WHM, if you see something like 0 idle workers, then this means that you will need to increase your maximum number of clients. This can be done from WHM as described here.

  8. Are you using any kind of caching on your website?

    If your website is gradually slowing, then you should consider using caching on the website. You should start with enabling Conservative Caching in the global configuration settings of your Joomla website, and then, if that doesn’t help things tremendously, you should enable the System – Cache plugin in your Plugin Manager (note that you should only do this as a last resort, or if you can’t afford to hire some experts to optimize your Joomla website, since the System – Cache plugin typically causes many conflicts and erratic behaviors). Also, make sure you enable template-specific caching if available (heavy templates usually have their own caching mechanism that has to be enabled separately).

  9. Do you have the necessary horsepower on your server?

    A few weeks ago, we have worked on a super large Joomla website which resided on a very old box with only 8 Gigabytes of RAM and a not so powerful quad core processor. That website, in our opinion, should reside on a very powerful server with at least 32 GB of RAM and a 48 core processor. We did manage to optimize it, but it was a costly operation for the client (who was informed in advance that the website was running on an inadequate server). Sadly, there is no secret formula that can tell you what kind of processing power (and RAM) is needed based on your number of pages and your number of hits because there are so many variables and unknowns that must be accounted for in such a formula (which, again, doesn’t exist). So, the only way to know that you need a bigger server is that if you notice that your RAM and your swap is entirely allocated almost all of the time and/or your CPU load is constantly high. By the way, an SSD drive is always a good idea as it speeds up filesystem access very noticeably (especially when it comes to write activities).

  10. Have you checked your MySQL slow query log?

    The MySQL slow query log is an indispensable tool to locate bottleneck queries. We use it in every Joomla optimization job that we get because it tells us the exact queries that are taking a long time to execute which helps us optimize these queries (or optimize the underlying tables/fields). When we work with the slow query log, we first address the queries that are taking a long time to execute, and then we attack the queries that appear in the log frequently.

  11. Have you checked that the long_query_time is set to 1 second in your my.cnf MySQL configuration file?

    If you examined the MySQL slow query log and saw nothing, despite the fact that slow query logging is enabled in the my.cnf file, then you should check whether the long_query_time setting, which defaults to 10 seconds, is set to 1 second in the my.cnf file. This will give you access to all the slow queries, not just those that take 10 seconds or more (it might be that you have a barrage of 1-second queries that are slowing down your website).

  12. Are you getting a sudden, unexplainable spike in your website’s traffic? It might be that your website is under a DoS (Denial of Service) attack which typically causes severe load issues on the server, and subsequently the website. You should check your logs, and, if you found anything suspicious, then you should consult with your host in order to block these attacks. There are also some extensions in Joomla that block DDoS attacks, but we don’t think highly of them since they are typically heavy and they often cause mysterious problems.

  13. Are you using all the extensions that you have installed on your website?

    The number of extensions that you have on your Joomla website is inversely proportional to its speed (and to its security). Surely, you are probably thinking, “but these extensions are doing nothing on the website and they have no data whatsoever”. That, of course, might be true, but many extensions come with plugins that may be running on each page load, affecting your page load speed. If you’re not using these extensions, then the best thing that you can do is to uninstall them!

  14. Have you done anything unusual on your website recently?

    Have you installed/uninstalled an extension? Have you updated an extension? Have you modified the ACLs? Have you asked a developer to work on your website? Have you modified the core in any way? Have you added something to your .htaccess file? Have you switched to another server/host or has your host upgraded applications on your server? (OK, that’s not really done on your website, but it’s definitely worth mentioning). If you have done any of the above, then your problem might just be there. Try, if possible, to revert to a backup to see if that resolves the problem.

  15. Is the problem happening on all the website?

    If the problem is just happening on one or a few pages on your website, then the slowdown might be caused by a module. Check the modules assigned to those pages and unassign them one by one until the problem is resolved (you will then be able to find the culprit module).

    If, after doing that, you still have slow load speeds on those particular pages, then the issue might be caused by a condition hardcoded in your template or it might be that these pages are using a different, slower template.

  16. Have you tried switching to a core Joomla template?

    Some templates, especially responsive templates, are heavy, very very heavy! If you’re using a template that you think is a bit heavy then try switching to a core Joomla template (such as a beez template) and see if that solves the problem. If you see a noticeable improvement after doing that, then don’t rejoice immediately, as it might not be the template, but it might be one or more modules with positions assigned in your original template but non-existent in the core Joomla template (please answer the previous question before answering this one).

  17. Have you checked your Google Webmaster Tools?

    Your Google Webmaster Tools can reveal some important information about your website when it comes to performance, particularly on when the performance problems started. This will probably give you a better idea on where to look to address the problem.

  18. Have you switched to a MEMORY based session table?

    The #__session table is probably the busiest table in any Joomla website, and, since the data in that table is not critical, then you can safely use the super fast MEMORY storage engine for this table. A discussion on how to do this is available here. That little change that you will make on the #__session table will yield exceptional results, trust us on this one (well, you can always try it without trusting us!)!

  19. Have you just upgraded/migrated your Joomla website?

    We don’t like to say it, but this is a fact: Joomla 1. 5 is faster than Joomla 2.5, and Joomla 2.5 is faster than Joomla 3.x, and, for some reason, we have a hunch that Joomla 3.x will be faster than Joomla 4.x. So, if you just upgraded/migrated your Joomla website then expect a degradation in the overall site’s performance until you perform some much needed optimization (that is the case especially when you’re running a high traffic and large Joomla website).

And, last but certainly not least…

  1. Have you contacted itoctopus?

    If you haven’t contacted us yet, then don’t be a stranger! We will definitely discover what the issue is and we will get your website running smoothly affordably, quickly, and cleanly! We’re always here for you – and for your Joomla website.

Creating a New Article in Joomla Taking a Long Time? Read This!

We had a client today complaining that when he clicks on the New button in Joomla’s Article Manager page, he has to wait for a long time for the Add New Article page to appear (the problem also happens when he clicks on an existing article in the Article Manager page to edit it). We did a quick analysis which revealed that the page was taking about 8 seconds to load, which is not normal since this page should take less than a half a second to load under normal circumstances.

Naturally, the first thing that we did was checking the slow queries in the MySQL Slow Query Log, but, alas, no slow query was logged there. This meant that the cause of the slowdown might be a hack, a rogue extension, or a batch of very similar queries being executed at the same time (typically in a while loop). We felt that this wasn’t going to be an easy task!

Luckily, we were wrong, since the first thing that we did was checking the HTML code generated on the New Article page (in order to see whether there was a malicious JavaScript there), and while doing that, we noticed one thing, there were 7,000+ tags being loaded into that new page! Naturally, we suspected that it was the huge number of tags causing the slowness, and so we completely removed the Tags field from the page the following way:

  • We opened the file article.xml located under the administrator/components/com_content/models folder.
  • We removed the following code:

    <field name="tags"
    	type="tag"
    	label="JTAG"
    	description="JTAG_DESC"
    	class="span12"
    	multiple="true"
    >
    </field>

  • We saved the file and uploaded it back.

  • We tested the Add New Article page, and, surprise surprise (well, not really a surprise, let alone two surprises), the page loaded in less than a second as it should be! (Removing the above XML content ensured that the Tags input box will no longer be loaded.)

Aha! So the reason was the huge number of tags! But of course, we can’t tell our client, who heavily relies on tags on his Joomla website not to use them, so we had to find an alternative solution to optimize the loading of this page, and so we did, and here it is:

  • We opened the file tag.php located under the libraries/cms/form/field folder.
  • We changed the following code:

    $query = $db->getQuery(true)
    	->select('DISTINCT a.id AS value, a.path, a.title AS text, a.level, a.published, a.lft')
    	->from('#__tags AS a')
    	->join('LEFT', $db->qn('#__tags') . ' AS b ON a.lft > b.lft AND a.rgt < b.rgt');

    to:

    $query = $db->getQuery(true)
    	->select('DISTINCT a.id AS value, a.path, a.title AS text, a.level, a.published, a.lft')
    	->from('#__tags AS a');

  • We removed the following lines:

    // Prepare nested data
    if ($this->isNested())
    {
    	$this->prepareOptionsNested($options);
    }
    else
    {
    	$options = JHelperTags::convertPathsToNames($options);
    }

  • We then saved the file and uploaded it back.

  • We reverted the changes we did on the aforementioned article.xml and we monitored the loading page and it still loaded in under a second! The problem was solved and we were able to keep the Tags field!

We probably have mentioned this before, we don’t like tags (in fact, we did, we did mention this before)! And that’s just another reason why we don’t like them and whey they should be avoided, yet people still use them and some extensions also use them (in a subtle way – in fact, the tags on our client’s website were generated by HWD MediaShare, which is a 3rd party media extension). But, if you’re stuck with them and it’s taking your website a long time to load the Add New Article page, then try the above solution and hopefully it should work for you. If it doesn’t, or if the solution presented here sounds a bit intimidating, then please contact us. We will solve your problem quickly, professionally, and for a very affordable fee!

How to Minimize Writes to Joomla’s Session Table to Reduce MySQL Load

Note: The solution presented in this post requires a core modification. As you may probably know by now, core modifications are delicate and may be wiped out with the next Joomla update. Please proceed with caution if you elect to implement the below.

If you’re running a high traffic Joomla website, then you probably have already noticed that a huge number of queries are actually updates to the #__session table. This is because every page load on the website creates/deletes/updates a row in the #__session. Naturally, these heavy write activities (again, we are talking about websites with a considerable amount of traffic) may cause load issues on the server.

Of course, you can always switch the #__session table’s storage engine to Memory instead of MyISAM or InnoDB, which is a good practice that we encourage, but this won’t solve the issue completely, because all these write activities, even on Memory tables, can still cause load issues (this is because every time any table is updated, all the MySQL query cache related to that table in any way will be wiped out).

Now, in order to reduce write activities to the #__session table on large websites, we have devised a quick, but very efficient, solution that requires a core modification. Here it is:

  • Open the file database.php located under the libraries/joomla/session/storage folder.
  • At the very beginning of the method write (which is defined around line 68 in Joomla 3.4.4) of the aforementioned database.php file, add the following code:

    if (JRequest::getVar( 'view' ) == 'article' || JRequest::getVar( 'view' ) == 'category'){
    	return true;
    }

  • Save the file and upload it back.

Once you do the above, you will notice a slight-but-not-very-slight drop in the load of the database server, since Joomla will no longer write to the session table when someone is just viewing an article or a category. Not too bad for a couple of lines of code, huh?

If you want to implement the above solution and you need help (or if you implemented the above solution and you noticed that it didn’t work for you), then please contact us. We will implement it for you for a very affordable fee, we’ll make sure that it really works, and you will gain new friends!

Byte Order Mark (BOM), Joomla, and Why You Can’t Login to Your Website

Late last night, a new client called us and told us that he can’t login to his Joomla website; that he’s being redirected to the login page with no errors when he logs in with either correct or wrong credentials. Of course, we’ve seen this issue several times, and so we thought it was an easy task, but we were wrong.

The problem with this task is that there were several issues causing the problem, and, if that wasn’t enough, there was a unpredictable twist that made our eyes bleed (they still are very red!). Let’s start with the normal issues:

  • The log and the tmp folders defined in the configuration.php file were wrong. We suspect the cause of the issue was a change in the environment (we don’t really like it when hosts change environments on Joomla administrators without telling them).
  • Apache was not able to write to the current session path, so we explicitly forced Apache to use another folder to save session data to by adding the following code (note that the below code is specific to Windows and is specific to our client’s filesystem):

    session_save_path(‘C:\\sites\\webroot\\joomla\\tmp’);

  • The session.php located under the libraries/joomla/session/ folder had the session_start line commented out. We really have no idea how that happened, but we think it was a developer who was trying to solve the problem but ended up creating a bigger problem. The reason why we think so is because when we uncommented this line out, we started seeing the following error on every page:

    php – Warning: session_start() [function.session-start]: Cannot send session cookie – headers already sent by (output started at configuration.php…).

After doing all the above, we still had a problem logging in, but the warning generated by fixing the last issue gave us a hint on what the idea was: the configuration.php file was outputting something when it shouldn’t (any output prior to initializing the session will break it and will cause the session to being re-initialized on every refresh, causing the login to fail). The problem is, we checked, double-checked, triple-checked, zillion-checked the configuration.php file and we couldn’t find any issue: there was no trailing space neither before the opening PHP tag nor after the closing PHP tag.

So we started banging our heads against every wall in the office, and between each bang, we checked the configuration.php file one more time to see if the problem is there. And then, suddenly, a mix of luck and inspiration merged into a huge lightning that hit us, making us notice that the configuration.php file was saved in UTF-8 format, with signature. Ah ha! So the configuration.php file had a Byte Order Mark (BOM) that was causing the issue.

We saved the configuration.php file in UTF-8 format, without signature (using EmEditor, which is, in our very humble opinion, the best editor out there) and that fixed the problem.

But what is a “Byte Order Mark”?

Without getting into too much details, a Byte Order Mark (BOM) is an invisible unicode character (which is, if you really need to know, U+FEFF) that is added to the beginning of a file to let the application using the file know which byte order the file is using. When you save a UTF-8 file with signature, then the BOM is added to the beginning of the file.

But why was the BOM added to the configuration.php file?

The BOM was added to the configuration.php file because our client had Thai characters in that file (specifically in the $MetaDesc, $MetaKeys, and the $MetaTitle values of that file), and used the host’s File Manager to save that file, and that File Manager was saving files with UTF-8 characters with signature (which it should have saved such files without signature). By the way, Notepad saves any UTF-8 encoded file with signature, which is why you should use an editor such as, ahem, EmEditor (we swear we are not making money every time we mention that editor) which allows the user to save a file in UTF-8 without signature.

If you’re having login issues to your Joomla website and you’re not seeing errors, then check, in addition to the methods described in the posts linked to above, that your configuration.php is saved in UTF-8 without signature (if it is already saved in UTF-8). If you’re still having problems, then your best option would be to contact us. We’ll solve the problem rapidly, affordably, and efficiently, and we promise you that we will be your friends as long as we live!

Large Joomla Websites Need Powerful Servers

For the past few days, we were working on optimizing a super large Joomla website. No, make that mega-large: we’re talking about a website that gets 100,000 uniques and around 700,000 impressions every day (yes, that’s every day). The website in question is a news website that is ranked in the top 20 websites in its respective country. The website was just migrated from Joomla 2.5.28 to Joomla 3.4.4, and it wasn’t working post migration. Needless to say, it was both a stressful and exciting experience to work on such a website.

We did many, many things to make it work. The most important things that we did were the following:

  • Disabling PHP session write activities for guests. This was done by adding the following line to the very beginning of the function write in the file database.php located under the libraries/joomla/sessions/storage folder:

    $app = JFactory::getApplication();
    if ($app->isSite())
    	return true;

  • Optimizing the articles.php model file located under the components/com_content/models folder as described here (note that this guide still works beautifully for Joomla 3.x).

  • Other optimizations.

The problem was that despite all our above efforts, we were only successful in making the website work for like 10 minutes before MySQL crashing. There was significant progress (the website wasn’t even working at all), but still, the website was still crashing. Obviously, we needed to do something drastic. So we followed this process:

  • We allowed access to the website.
  • The MySQL slow query log was getting filled with slow queries.

  • We blocked access to the website.

  • We optimized the slow queries that we found in the log.

  • We cleared the log.

  • We repeated the above process until the website worked.

Once we finished the work, we started thinking: yes, it’s a huge website with significant traffic, but we worked on similar websites before, and they weren’t that hard to optimize. What was different this time?

It didn’t take us long to discover the difference between this website and other similar websites that we worked on: the website resided on an old server with 8 gigabytes of RAM, a 4 core processor, and an old, slow hard disk. To put things in perspective, our work laptops have 16 gigabytes of RAM, 4 core i7 processors, and SSD drives! Which means that our laptops are much faster than the server powering a website that gets about 3 million uniques every month!

After finishing the job, we communicated our findings to our client, and we told him that most of the optimizations that we did were not really needed had the website resided on a powerful server. In our opinion, that website needs at least a 64 core server with 64 gigabytes of RAM and an SSD drive to work smoothly. The fact that it is working smoothly now is (very humbly) pretty impressive.

The message that we want to convey in this post (besides the one where we’re bragging about our 16 GB of RAM Japanese laptops) is that large Joomla websites really need powerful servers (we know, duh), and sometimes, Joomla is not the real culprit behind a website’s poor performance. So, if your website is running slow and you’re not sure whether you need to optimize Joomla or upgrade your server, then let us know. We will give you an honest assessment of your environment, letting you know whether you need to update your server, optimize Joomla, or both! Oh, and we won’t charge you much!

How to Add the Google Tag Manager’s Code to Your Joomla Website

We are getting an increasing number of requests asking us to add the JavaScript code for the Google Tag Manager (which is another tracking tool mainly used by marketing departments). We have decided, in this post, to share a quick and simple guide on how to do this.

First thing you will need is to ensure that you have, in the index.php file located under your templates/[your-template] folder, the following code just under the <jdoc:include type=”head” /> line:

<jdoc:include type="modules" name="code" />

If the above line is not there, then you should add it, and then save the index.php file and upload it back to your website.

The next step would be to create an override for the Custom HTML module’s layout as described here in order to remove the unnecessary div tag (which will, unsurprisingly, break the HTML compliance of your website since you can’t have a div tag in the head section in standard HTML).

Once that is done, you should create a Custom HTML module with the following content (make sure that your editor allows you to type in JavaScript code):

<!-- Google Tag Manager -->
<noscript><iframe src="//www.googletagmanager.com/ns.html?id=[YOUR GOOGLE TAG MANAGER ID]"
height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>
<script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
'//www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','[YOUR GOOGLE TAG MANAGER ID]');</script>
<!-- End Google Tag Manager -->

(You will need to change [YOUR GOOGLE TAG MANAGER ID] with your own).

You should then assign the module to all the pages, and you must ensure that its position is set to code (you should hard type code in the Position field even if it’s not in the dropdown). You must also ensure that, in the Advanced Options tab, the Alternative Layout is set to nodiv (that is if you followed the above guide on overriding the output of the Custom HTML module).

Now, there is just one small thing to do, which is completely clearing the cache on your Joomla website (as well as that of your template, if it has its own caching mechanism).

If you followed the above instructions, then go ahead and show the world your beautiful smile of success because your website has now the Google Tag Manager installed!

Of course, Google Tag Manager is a vast realm, and it’s possible that you will need to send Google some specific information about your website’s content. If that is the case, then feel free to contact us. We’ll be happy to do this for you professionally, cleanly, and without taking much of your money

Hit Tracking Not Working when Joomla’s Caching Is Enabled: How to Solve

Joomla’s issues caused by caching are many, but one of these issues is extremely annoying and we are surprised how the Joomla development team hasn’t come up with a solid solution to it yet. In case you’re still wondering what the issue is, then it is no other than the conflict between hit counting and caching. Here’s the problem in details:

  • You enable caching on your website (any type of caching – even Conservative Caching).
  • Joomla caches the content item or the whole page (depending on the type of caching you apply, System Caching will cache the whole page, and Conservative Caching will just cache the content item).

  • When a visitor visits a page, then Joomla checks first if the there is a cached version of the page or the requested content item, and if there is, then Joomla will serve the cached version, if not, then Joomla processes the request, serves the page, and then creates the cached version of that page/content item. If the page/content item is already cached (which is the case in about 99% of the visits), then Joomla will not count the visit as a hit as the script that runs to increment the number of visits does not run on a cached version of the content item or the page.

If you’re a technical person, then you will know that it is really hard to solve the above problem without any core modification since the problem lies in the core (if you’re not a technical problem, then you should just take our word for it). However, at itoctopus, we have devised a method that will address the caching problem without core modifications. Here’s how:

  • Open the file default.php under the templates/[your-joomla-template]/html/com_content/article folder.
  • At the end of the file, add the following two lines:

    <script src="//ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
    <script>$.get( "<?php echo(JURI::root()); ?>updateHits.php", { articleId: "<?php echo($this->item->id); ?>"} );</script>

    (Note that the first line is not needed if you already have jQuery loaded.)

  • Upload this file (after extracting it) to the root folder of your Joomla website.

  • Clear your Joomla cache.

  • That’s it! That’s really it! Now your website can track the hits of your cached items.

But, how can the above method solve the problem?

The above method solves the problem because the cached content item/page will always contain the JavaScript code that will increment the number of hits, and so that code will always run even if the content item/page is cached.

But, does the above method provide 100% accuracy?

No – and we don’t think anyone out there needs to track the number of hits with a 100% accuracy since many hits come from bots (even on low traffic websites). In any case, the reason why the above method doesn’t provide 100% accuracy is because the number of hits will be incremented twice on non-cached items: 1) by Joomla’s core, and 2) by our code above. Of course, the margin of error in detecting hits is negligible but there’ll still be a margin of error.

If you want 100% accuracy when tracking hits through your Joomla website, then you should consider disabling hit tracking (note that doing that will mean a change to your Joomla core).

But, why should anyone care about the number of hits in Joomla?

Well, if you have a module that lists your “most read articles” on your website and your hit tracking is not working properly because of caching, then most likely the information displayed in the module will be wrong (your important articles will not be displayed, and your non-important articles will be displayed – at the very least the ordering of your most important articles will be wrong).

Does the above method work on K2?

No – but K2’s hit tracking doesn’t have any conflicts with caching. So you shouldn’t need it for K2.

Now, if you’re running into the same dilemma as the majority of Joomla administrators out there (e.g. hit tracking is not working when you have any kind of caching turned on), then try implementing the above – it should solve your problem and your Joomla website should track your articles’ hits reliably. If you think the above is somehow in the advanced realm (or maybe in the twilight zone), then fear not… Just contact us and we’ll do it for you for a very, very affordable fee. Note that a sufficient mix of love, friendliness, and professionalism is always added for free!

“Possible IE XSS Attack found.” when Trying to Upload a File in the Media Manager

Note: The solution presented in this post requires, ahem, a slight modification in a core file. Proceed with caution, and keep in mind that any change you make may be no more when you update your Joomla website.

A couple of hours ago, a high level developer working for a major client of ours told us that she was getting the following error while trying to upload an HTML file in the Media Manager:

“Possible IE XSS Attack found.”

Without giving the issue much thought, we told her to do the following:

  • Login to the backend of the Joomla website.
  • Click on Content -> Media Manager.

  • Click on Options on the top right.

  • Next to Legal Extensions (File Types), add html as a legal type.

We thought that the problem lied in the fact that files with a .html extension are not allowed by default in Joomla, and that they must be explicitly allowed in the Media Manager‘s settings. While that was true, it didn’t solve the problem…

A few minutes later, she emailed us telling us that she already did that. So we investigated the problem much more thoroughly, and we discovered something new, something that we didn’t know of before: Joomla, for security reasons, doesn’t allow any file that contains HTML tags to be uploaded, and that is enforced in the file media.php, which is located under the libraries/cms/helper. If you read that file (start at line 190), you will know that Joomla scans the uploaded file for most known HTML tags and immediately returns the error if any tag is found. What makes this security rule even more aggressive is the fact that it doesn’t care about the file type: even if the file is called myimage.jpg it is checked for HTML tags. Hmmm…

Now, since the users using the backend were trustworthy, we were able to disable this security rule only for the backend. We did that the following way:

  • We opened (using FTP) the file media.php located under the libraries/cms/helper folder.
  • We searched for the following line:

    $xss_check = file_get_contents($file['tmp_name'], false, null, -1, 256);

    Just above it, we added the following line:

    $app = JFactory::getApplication();
    if ($app->isAdmin())
    	return true;

  • That’s it! Doing the above allowed the developer to upload files with HTML content to the website. The problem is solved!

But what’s with “IE XSS” error?

We know, the error doesn’t really make any sense when you take into consideration that the developer was not even using IE (and, of course, was not experimenting with XSS attacks against her employer’s website). We suspect that this security rule was added because of an exploit that is specific to Internet Explorer, hence the name (by the way, this security rule didn’t exist in versions of Joomla older than 3.2.0).

If you’re seeing the same problem while trying to upload an HTML file to your Joomla website using the Media Manager, then try the above solution, it should work. If you’re facing this problem in the frontend, then you will need to just add return true; (instead of the 3 lines above), but keep in mind that you may be opening your website to security exploits, and we don’t recommend it. If you’re in a dilemma as to whether you should implement this solution or not, and you need help, then please contact us. We are super experienced in Joomla, our work is super clean, our rates are super cheap, and we are always super ready!

How to Change Joomla’s SEF Suffix From “html” to “php”

Note: The solution presented in this post involves a modification to a core Joomla file. Please proceed with caution and keep in mind that a core modification can be wiped out with a future Joomla update, potentially breaking your website (or, at best, breaking your website’s links).

Every now and then, a client calls us and tells us that his company wishes to have the suffix of their website’s links to be something other than html. Usually, the client wants to change the suffix from html to either php or htm. Nearly every time we had this call, the client expected to be able to do this by changing a setting in Joomla’s backend. Of course, the first thing that we do is that we tell the client that this is not possible – and that core modifications are required in order to do that. We then get approval from the client to implement the solution for him and we proceed to work!

Now, since we are at itoctopus super-generous with an obsession of knowledge sharing, we are going explain the process of changing the suffix on a Joomla website from html to php in this post. Here’s how:

  • Open the file site.php which is located under the libraries/cms/router folder.
  • Find this line (should be line 110 in Joomla 3):

    $vars['format'] = $suffix;

    Replace it with:

    $vars['format'] = 'html';

  • Find this line (should be line 144):

    if ($format = $uri->getVar('format', 'html'))

    Replace it with:

    if ($format = $uri->getVar('format', 'php'))

  • That’s it! That’s really it! There’s nothing more to do!

This seems super easy! Is there a caveat?

Well, actually there is. The problem with the above method is that you will have duplicate links, for example, both http://www.[your-joomla-website].com/page-1.html and http://www.[your-joomla-website].com/page-1.php will work (obviously, Google will not like that). However, the solution to this problem is very straightforward. All you need to do is to add the following line to the beginning of your .htaccess file:

RedirectMatch 301 (.*)\.html$ http://www.[your-joomla-website].com$1.php

The above line will solve the problem.

Will the above method work for a suffix other than “php”?

If you want to change the suffix of the links on your Joomla website from html to anything else, then just replace php in the code above to the suffix of your choice. You need, however, to make sure that you add the following line to your .htaccess file (to tell Apache to process any URL with an unknown suffix as HTML):

DefaultType text/html

If you need help implementing the solution above, then look no further as we are here to help! Just contact us and let us prove to you how efficient, knowledgeable, and affordable we are!

Joomla 3: No Save Button on “Add Contact” Page

One of our very large clients had a weird problem on a newly migrated Joomla website: everything was working properly on the website, except for the fact that they couldn’t add any contact (in the Contacts component) because the “Save” button wasn’t there. In fact, the only button that they could see was the “Cancel” button.

Although this was the first time that we saw this problem, we had a hunch that its root cause had something to do with the assets table. So, we clicked on the Categories tab in the Contact Manager,and then we clicked on the Rebuild button on the top right, with the hope that that would solve the problem. Unsurprisingly, it didn’t. We knew it wasn’t that simple, but we still believed that the problem lied somewhere in the assets table.

So we logged in to phpMyAdmin and checked the #__assets table there, and we immediately realized what the problem was. There wasn’t a single entry referencing any contact category in that table. It seems when the website was migrated, the references to contact categories where not created in the #__assets table.

So, how did we fix the problem?

Fixing the problem was super easy. All we needed to do was to go back to Joomla’s backend, open each and every contact category that the website had, and then save it (saving the contact category [and any Joomla category] creates the right entry in the assets table).

If you have the above problem on your website (in the Contacts extension or in any other extension), then try rebuilding the categories and re-saving them. Most likely this will fix the problem. If it doesn’t, then the problem might be somewhere else, and you might need some Joomla experts (like, ahem, us) to handle it. Just contact us and we’ll address the problem for you in as little time as possible and for a super affordable fee!

How to Password Protect the Backend of a Joomla Website with .htpasswd (and .htaccess)

Early last week, we had a case where a malicious extension was subtly installed on the Joomla website for one of our clients. Luckily, our extra security measures prevented actual harm, but still, we were confident that if the person who installed that extension was a bit more persistent, he could have wiped out the entire website.

What was surprising, though, was not the fact that that person was lazy (generally malicious users are very, very lazy), but it was the fact that he was able to install the malicious extension on a super protected Joomla website. We needed to find out how it was done to prevent it in the future…

So, we examined the logs thoroughly, and it didn’t take us log to find out how that malicious extension was installed: it was installed through Joomla’s backend. So, our next job was to check how the user gained access to the backend, and it also didn’t take us long to find out that he logged in with a valid username and password, which meant that the username and password of one super user were compromised! What was even worse was that there was no evidence that the malicious user tried to login through brute force, which meant that he knew the username and password in advance! A horrible thought, probably more horrible than the reaction of Jimmy the Tulip after realizing that there’s mayonnaise in his burger! Why is it that horrible, you might wonder? Well, because you can’t do much to protect a website with a compromised set of credentials other than just changing the password!

But we decided to do more, we decided to protect the website with an additional login: the .htpasswd login! In case you’re wondering what it is, the .htpasswd login is an Apache based login that will protect a specific directory on the website with a username and a password. Here’s how we did it:

  • We generated the .htpasswd file using an online tool (just google “generate .htpasswd” and typically the first link is what you want). You can also generate the htpasswd file from the command line if you have root access to the server hosting your Joomla website. (Note: We re-generated the .htpasswd using the command line for that website as we weren’t very comfortable using a set of credentials that we generated using an online tool).
  • We copied the .htpasswd file to the /home folder on the server and we ensured that Apache (e.g. the Joomla website) has read access to that file.

  • We created an .htaccess file under the administrator directory of the Joomla website and we added to it the following lines:

    AuthName "Authorisation Required"
    AuthUserFile /home/.htpasswd
    AuthType Basic
    Require valid-user
    ErrorDocument 401 "Authorisation Required"

  • That’s it! Now anyone who wants to login to the website has to provide an additional set of credentials!

We know – this is ridiculously easy! What’s even more exciting is that any brute force attack will be automatically blocked by the CSF (ConfigServer Security and Firewall) firewall after (usually) 5 failed attempts.

But how did the malicious user know the username and the password of the super user?

We’re not exactly sure, but we’re inclined to say it was because one of the major staff members had a virus on his machine, which probably transmitted the Joomla credentials to the malicious user. Of course, you might say that the same can happen for the .htpasswd credentials, and our answer is “you’re correct”, but the company implemented a policy of daily scanning their entire network of PCs ever since the incident, immediately disconnecting any infected machine.

We hope that you found this post useful! If you did, then we did a good job! If, however, you feel that you need help password protecting your Joomla website with an .htpasswd file, then you can just contact us. Our rates are affordable, our work is quick and professional, and we love, we really love, our clients, and our clients love us!

Unable to Delete Items in Joomla’s Trash

Note: This post applies to Joomla 2.5 only. If you’re running Joomla 3.x and you can’t delete articles from Joomla’s trash, then please let us know in the comment section of this post as the below solution will not work for you.

Yes – we know – our regular readers might say that we have discussed this before. But back then, we only thought that the inability to delete items from Joomla’s trash was a user error, and not an application error. Obviously, we were wrong – which explains why we are writing another article on the same exact subject!

So, earlier this week, a new client with a Joomla 2.5 website called us and told us that he’s not able to empty Joomla’s trash, even after ensuring that he filtered for “Trashed” articles before trying to empty the trash. We checked his website and we were able to easily verify the claim. Interesting…

So, we dug deeper and we found out that the issue was caused by the fact that many of the trashed articles did not have an entry in the assets table. So, just opening and saving one of those trashed items allowed us to remove it completely from the trash (note that when you open and save an item, Joomla re-creates its entry in the assets table if that entry is not already there).

But the problem is that our client literally had 20K trashed articles that need to be deleted and the absolute majority of these articles didn’t have an entry in Joomla’s assets table. Of course, resolving this problem is as easy as running the below query on the #__content table:

DELETE FROM #__content WHERE `state`='-2';

(Where #__ is the database prefix of your Joomla website as defined in its configuration.php file).

But doing the above is not what our client wanted, since he didn’t want to call us to run the above query each time he wanted to empty his trash. Additionally, executing the above query will affect the data integrity in the assets table, since some content item entries will remain there, but with no actual articles to be associated with.

So, our next job was to find where Joomla is checking for the article’s entry in the assets table before deleting it… It took us some time to find where that check was done: it was in the table.php file (which no longer exists in Joomla 3.x) under the libraries/joomla/database folder. The check was done in the function delete of the aforementioned file, specifically in these lines:

if ($this->_trackAssets)
{
	// Get and the asset name.
	$this->$k = $pk;
	$name = $this->_getAssetName();
	$asset = JTable::getInstance('Asset');

	if ($asset->loadByName($name))
	{
		if (!$asset->delete())
		{
			$this->setError($asset->getError());
			return false;
		}
	}
	else
	{
		$this->setError($asset->getError());
		return false;
	}
}

So, in order to address the problem, we modified the above code to:

if ($this->_trackAssets)
{
	// Get and the asset name.
	$this->$k = $pk;
	$name = $this->_getAssetName();
	$asset = JTable::getInstance('Asset');

	if ($asset->loadByName($name))
	{
		if (!$asset->delete())
		{
			$this->setError($asset->getError());
		}
	}
	else
	{
		$this->setError($asset->getError());
	}
}

Note that the return false; statement was removed in the fixed code.

But isn’t that a core change?

Yes – it is. But we assume that if one is still running Joomla 2.5, then he should expect core modifications to address some bugs. If you’re not comfortable with doing the core change, then you can just set up a cron that will regularly delete articles with a state of -2. Note that if you would like to preserve the integrity of the assets table, then you you should also delete the associated entries in the assets table.

But why would a content item not have an entry in the assets table?

Typically, there are 2 reasons for this to happen:

  1. You are deleting the content item entries from the assets table in order to optimize the content creation process on your Joomla website as described here.
  2. One of your extensions is creating content items (articles) in the Joomla CMS without adding an associated entry in the assets table.

Now, if you only have a few articles that you can’t delete from the trash, then just open these articles, save them, and then try to delete them again. If you have thousands of articles that you can’t delete and you are running Joomla 2.5 (or 1.6, or 1.7), then try the core modification above; it should work for you! If you’re not comfortable with core modifications, then just contact us and we shall implement an alternative solution for you in very little time and for a very small fee!

Why It’s Important to Have Contiguous IDs in Joomla Tables

One of our newer clients called us yesterday morning and told us that his Joomla website was running very slow. So we checked his website and we immediately knew the cause of the problem, Mosets! If you are a regular reader of our blog, then you probably know that we don’t think highly of this extension – not only because it’s unoptimized by default, but because its code is super messy. But we’re not here to whine, we’re here to fix problems!

So we indexed the should-be-indexed-but-not fields in Mosets’ tables, and that changed things dramatically (in a positive way, of course). However, the website was still running slow, not as slow as before, but still, it was noticeably slow. So, again, we checked the slow query log, and we noticed that there are still many slow queries related to Mosets. Hmmm…

A closer examination of these queries revealed the cause of the slowness, which is the LIMIT command in MySQL. You see, that Mosets instance had over 300k rows in its main table, and when the LIMIT command was used on that table for higher ranges, the slowness occurred. The cause of this problem is the way MySQL handles the LIMIT command (it scans the table until it reaches the starting position of the LIMIT, which means that the higher that starting position is, the more time the query will take to execute).

So, how did we solve the problem?

Usually, when we have a performance bottleneck caused by the MySQL LIMIT command, we add a filter to the query that will start searching only the rows with IDs higher than the starting limit in the LIMIT command. In other words, we modify this query:

SELECT * FROM #__table LIMIT 50000, 30

to this one:

SELECT * FROM #__table WHERE id >= 50000 LIMIT 0, 30

The second query is much, much faster than the first one – but there is one major caveat: The IDs have to be contiguous (e.g. there shouldn’t be any gaps between the IDs of consecutive rows) and the first ID must be 1, because if this is not the case, then the optimized query will yield wrong results.

In order to solve this problem for our client we created a PHP script that made the IDs of the offending table contiguous, with the first ID starting at 1. Of course, the job wasn’t very simple there were other tables using these IDs, and because the content of the Joomla website was also using the old IDs. What made things even more complex was that the links too had to be redirected to the new links (since the old links contained the IDs). Needless to say, it was a very painful experience but the rewards were enormous: the website now runs at lightning fast speed (we did some other optimizations as well) and the client was exalted!

Now, Mosets was just an example, but any table in Joomla (whether a core table or a non-core one) can cause the same issues when using LIMIT commands on higher ranges. So, if you need help making the IDs of a specific table contiguous, then look no further! We have done it before and we know we can do it for you! Just contact us and rest assured that we won’t charge you much and that you will be a very, very happy client!

How to Display the Time for the Last Logged In Users

A client called us earlier this week and told us that when he logs in to his Joomla website and he checks which users are logged in to the website, he only sees the login date. Given that all his users login nearly every day, just seeing the date wasn’t enough for him. He also wanted to see the time at which these users logged in. He said that previous versions of Joomla displayed the date and the time, but Joomla 3.x did not!

In all fairness, we didn’t notice this issue until the client told us about it. So, we logged in to the backend of his Joomla website and then went to the Extensions -> Module Manager, and then selected Administrator from the left dropdown (directly under Filter), and then searched for the module Logged-in Users (this module is the one responsible for displaying this information). To our disappointment, the module didn’t have a setting to show the time, but rather it only allowed us to change 2 things: 1) how many logged in users can be displayed at once, and 2) whether we wanted to display the username or the name of the last logged in users.

But, at itoctopus, we always find a solution to everything. So, in order to accommodate our client’s request, we opened the file default.php located under the administrator/modules/mod_logged/tmpl and we searched for this line:

<i class="icon-calendar"></i> <?php echo JHtml::_('date', $user->time, JText::_('DATE_FORMAT_LC4')); ?>

(Note: The above line could also be: <i class="icon-calendar"></i> <?php echo JHtml::_('date', $user->time, 'Y-m-d'); ?> in previous versions of Joomla 3).

We then changed it to the following line:

<i class="icon-calendar"></i> <?php echo JHtml::_('date', $user->time, 'Y-m-d H:i:s'); ?>

We checked the website and, unsurprisingly, it was now displaying both the date and the time (to the second!). The problem is solved!

But, are there any caveats?

Well, unfortunately, there is this one small caveat. See, the above is a core modification, which means that if you update your Joomla website, the module may be reverted back to what it was (e.g. displaying only the date, and not the time). In order to avoid that, you will need to override the layout by copying the aforementioned default.php from the administrator/modules/mod_logged/tmpl folder to the administrator/templates/[your-joomla-admin-template/html/mod_logged/ folder and modifying it there. This will ensure that any updates to your Joomla website will not overwrite your changes.

If you want to display the date and the time of the last logged in users on the homescreen of the backend of your Joomla website, then all you need to is to follow our super simple guide above. If you think it’s a bit on the technical side and don’t want to do it yourself, then go ahead and contact us. Our rates are super affordable, our work is super professional, and all of us are super duper friendly!

How We Fixed a Very Weird Blank Page Problem on a Joomla Website

A large company (that we have worked with before) emailed on us on Friday morning and told us that they were seeing a blank page on their homepage. When we visited the website we noticed that the page was almost blank. In fact, there was some content on the website. But the main component and all the modules were not showing. Hmmm…

The backend was working and so we logged in and we set the error reporting to Maximum in the global configuration settings. We then cleared the cache and checked the website but the problem was still there. Yet another hmmm…

So we followed our guide in order to ensure that all the errors were displayed, but still, the same blank page appeared. Again, the page wasn’t entirely blank, a JavaScript widget appeared on the bottom right (that JavaScript widget was actually a plugin).

We started suspecting the plugins – and so we switched the template to another template in order to see what happens, and what happened was that the website worked, and this is a definitive sign that the problem is not caused by a plugin (since the same enabled plugins run on all templates).

After switching back to the original template, we realized that we were left with 3 suspects: The component the home menu item points to, one of the modules, or the template. The component that was used on the homepage was a core Joomla component, which is the com_content component (the home menu item was pointing to the “Featured Articles” view). Additionally, none of the other pages were working (and some of the other pages were using different components) so we dismissed the component as a potential suspect. So, we moved on to the next suspect: the modules. The modules were actually quite easy to rule out, as the only thing that we needed to do was to run the following query in phpMyAdmin after selecting the database that powers the Joomla website:

UPDATE `#__modules` SET `published` = 4 WHERE `published` = 1 AND `client_id` = 0;

As you can see in the above query, we set all the published field of all the enabled modules to 4 (we didn’t want to set it to zero because if we did that, then we will re-enable the intentionally unpublished modules once we re-enable the modules that we just unpublished).

We then visited the website and it still didn’t work despite the fact that we disabled all the modules (we re-enabled the modules that we disabled using the following query: UPDATE `#__modules` SET `published` = 1 WHERE `published` = 4 AND `client_id` = 0; ). This meant that the problem was definitely the template. A closer look at the template revealed that it was a Helix template with some corrupt settings. So we asked the system administrator to provide us with a previous backup copy of the database, and then we copied the template’s settings from the #__template_styles table from the backup database to the #__template_styles table on the new database. That fixed the problem!

But why did the problem happen in the first place?

We don’t know what caused the data corruption to happen. We just know that it happened. We do suspect, however, that it was caused by a bug that is triggered when saving the settings to the template. Unfortunately, we were not commissioned to investigate further on the cause of the issue so we can’t give a definitive answer.

If you’re having the same problem on your website, then try following the above process in order to narrow down the location of the problem. If you think that our process is not that simple, or if you think it’s too technical, or if you just think that you need help, then please, contact us! We are here to help. Our rates are affordable, our work is neat and professional, and you can always, always rely on us!

Joomla Administrator Login Redirects Back to Login Page Without Errors

Yesterday, we had a new customer, from Denmark, with a very nice Joomla website, asking us to fix one of the most annoying Joomla problems. The problem was that whenever he tried to login from the backend, he was getting redirected back to the login page without any error. He was confident that the username and the password were correct and he was very troubled by the fact that he wasn’t able to login to his own website.

When the backend login redirects back to the login page with no errors, then this means that the problem is not with the credentials but somewhere else. Luckily, we did have the same exact symptoms before on another Joomla website and so we thought we’d be able to fix the issue instantly. But we were wrong…

The problem had nothing to do with a disabled plugin or with the session.save_path. The problem had to do with something else. So, what we did was that we set the $error_reporting variable to “maximum” in the configuration.php file (which is located under the root directory of the Joomla website) and we tried again. Of course, it still didn’t work, but this time we saw the following messages:

Notice: session_start(): Server localhost (tcp 20046) failed with: Connection refused (111) in /home/[user]/public_html/libraries/joomla/session/session.php on line 665

Warning: session_start(): Cannot send session cache limiter – headers already sent (output started at /home/[user]/public_html/libraries/joomla/session/session.php:665) in /home/[user]/public_html/libraries/joomla/session/session.php on line 665

Hmmm! It seems that for some reason, Joomla was trying to store the sessions in an alternative location (usually, Joomla stores the sessions in the database). So we took a closer look at the configuration.php file and we noticed the following line:

public $session_handler = 'memcache';

Aha! Joomla was trying to use memcache to store the sessions, but it was unable to (seemingly because of connection issues, or maybe because memcache doesn’t even exist on that server).

So, in order to solve the problem, we just switched the session handler to database, by changing the above line (again, in the configuration.php file) to:

public $session_handler = 'database';

And that fixed the problem!

But, what caused the session handler to switch to memcache?

We’re not 100% sure, but we suspect it was a user error (e.g. the Joomla super user mistakenly changed it to memcache).

So, if you’re trying to login to your website but you keep getting redirected to the login page (with no errors being displayed), then a good start would be to check the $session_handler variable in your configuration file. If that doesn’t help, then check the #__extensions table (in the database powering your Joomla website) and make sure that the appropriate plugins are enabled (mainly the plg_user_joomla). If that still doesn’t fix the problem, then we’re here to help. All you need to do is to contact us and we’ll fix the problem for you as soon as we can and for a really, really competitive fee!

“No Parent” Is Missing from the “Parent” Dropdown When Trying to Add a New Category in Joomla

A new client of ours called us today and told us that he’s facing a major challenge after migrating his Joomla website from 1.5.26 to 3.4.1. He told us that the No Parent option seems to be missing from the Parent dropdown in Edit Category mode, which meant that whenever he tries to add a new category anywhere on the Joomla website, that category must be the child of another category. In other words, he can’t have new top level categories on his website.

Of course, the first thing that we tried to do was to rebuild the categories by clicking on the Rebuild button in the Category Manager page. But, instead of fixing the problem, it threw the following error (by the way, the error was in green [which typically means success], instead of red – weird):

Failed rebuilding Categories tree data.

Hmmm! That error meant that there was corruption at the database level in the #__categories table. So, we logged in to phpMyAdmin, selected the database that powers the website, and checked the table #__categories. Unsurprisingly, we noticed that the ROOT item is missing.

So, what is the ROOT item?

The ROOT item (or the ROOT row, or the ROOT entry) is a row that must exist in all the tables where parenting is available (e.g. an item can be the child of another item – there are two core tables that must have a ROOT entry: the #__assets table and the #__categories table). That row must have the following characteristics:

  • It must have an ID of 1.
  • Its parent must be 0 (e.g. it doesn’t have any parent).

  • Its lft (left) value is 0, and its rgt (right) value is the maximum ID in the table + 1.

So, in order to fix the problem, we executed the following query to re-create the ROOT item:

INSERT INTO `#__categories` (`id`, `asset_id`, `parent_id`, `lft`, `rgt`, `level`, `path`, `extension`, `title`, `alias`, `note`, `description`, `published`, `checked_out`, `checked_out_time`, `access`, `params`, `metadesc`, `metakey`, `metadata`, `created_user_id`, `created_time`, `modified_user_id`, `modified_time`, `hits`, `language`, `version`) VALUES
(1, 0, 0, 0, 10000, 0, '', 'system', 'ROOT', 'ROOT', '', '', 1, 0, '0000-00-00 00:00:00', 1, '{}', '', '', '{}', 42, '2011-01-01 00:00:01', 0, '0000-00-00 00:00:00', 0, '*', 1);

The maximum ID, which is in bold red in the above query, is the result returned by executing the following query:

SELECT MAX(id) + 1 FROM `#__categories` WHERE 1

After doing the above, we tried clicking on the Rebuild button again, and, to our joy, we saw the following (success) message:

Categories tree data successfully rebuilt.

Yey!

We then tried to add a new category and, unsurprisingly, the No Parent option was available once again! The problem is officially solved!

But, what caused the problem in the first place?

We think that the problem was caused by the migration tool that our client used to migrate the data. We don’t want to disclose the name of that tool, but that’s one of the reasons why we have our own scripts for migrating Joomla websites.

In closing, if you have the same problem on your Joomla website, then check if the ROOT entry exists in the #__categories table. If it doesn’t, then you should re-add it using the above query. If it does, then the problem might be elsewhere – in that case, go ahead and contact us and we’ll try our absolute best to solve the problem for you quickly and cheaply!

Subqueries: A Major Cause for Performance Issues on a Joomla Website

We were commissioned yesterday to optimize a large Joomla website. That website was using FlexiContent as its content engine. In case you don’t know, FlexiContent extends Joomla’s core content management system, and does not replace it (unlike K2). The main reason why our client was using FlexiContent was that it supports multiple categories (per article). We aren’t hugely fond of that extension…

In any case, the website was super slow. So, as usual, we checked the slow query log and we noticed that the following query was taking some insanely long time to execute:

SELECT * from `#__flexicontent_content_hits` WHERE content_id < (SELECT max(content_id) - 200 FROM `#__flexicontent_content_hits`);

As you can see, the above query is technically a query within a query (which is called a subquery). While this works, it is super inefficient since MySQL is not the best RDBMS when it comes to subqueries (other RDBMS‘s fair much better when using subqueries).

But why is MySQL slow for subqueries?

MySQL executes all its queries outside-in (that’s the opposite of inside-out, in case you’re wondering, but you’re probably not). In other words, MySQL runs the outside query first, and then, it will execute the inside query for every single row returned from the outside query, which means that if the #__flexicontent_content_hits table has 4,000,000 rows, then this will technically mean that the above query is the equivalent of 4 million queries. Not good at all! In fact, we think that the client was lucky that his website still worked (albeit very slowly) considering the number of rows he had in that table.

So, what did we do to fix the problem?

In order to fix the problem, we split the query into two queries (the following is a pseudocode):

FIRST_QUERY_RESULTS = SELECT max(content_id) - 200 FROM `#__flexicontent_content_hits`;
SECOND_QUERY_RESULTS = SELECT * from `#__flexicontent_content_hits` WHERE content_id < FIRST_QUERY_RESULTS;

The above made a huge difference on the server’s load, and, ultimately, on the website’s speed. This is because the above method meant that the second query consisted only of one query, and not of n queries, where n is the number of rows in the `#__flexicontent_content_hits` table.

So, we have a message to all Joomla developers out there (and to all LAMP developers): Do not use subqueries, they create horrible performance issues and the alternative is very, very simple – just split your query into two and that’s it.

Now, if your website suffers from performance issues (that may or may not be related to subqueries), then just contact us and you can rest assured that your website will become quicker than ever with us. Keep in mind that both our very affordable fees and our love apply!

How to Display Page Titles from HubSpot on Your Joomla Website

This weekend we had a very interesting task. We were told, by one of our top clients, to display a list of all their Pages from HubSpot, along with a link to them. As usual, we obliged, and we were glad that they gave us this task, because it was something that we haven’t done before.

The first thing that we needed to do was to get used to and understand HubSpot‘s API – which we first thought would only work after creating a HubSpot application – but we later discovered that this wasn’t needed. All that we needed to do was the following:

  • Create an API key on HubSpot which can be used to authenticate the API requests. A HubSpot API key can be acquired from this link (note that you must be logged in to HubSpot before visiting that link).
  • Grab all the pages from HubSpot using the COS Page Publishing API.

  • Filter the pages (according to client-specific requirements that we won’t mention here).

  • Display these pages in a module on the website’s homepage.

Of course, the above steps are definitely not exhaustive, but they are the main steps.

So, what was the code that we used to grab the data?

We grabbed the data using the following PHP function (which we placed in a newly created module called mod_hubspot):

function getHubSpotPages($apiKey){
	$hubSpotJson = file_get_contents('http://api.hubapi.com/content/api/v2/pages?hapikey='.$apiKey);
	$data = json_decode($hubSpotJson,true);
	$hubSpotList = array();
	if (isset($data['objects'])){
		$data = $data['objects'];
		for ($i =0; $i < count($data); $i++){
			$item = new stdClass();
			$hubSpotList[$i]= $item;
			$hubSpotList[$i]->link = $data[$i]['published_url'];
			$hubSpotList[$i]->title = $data[$i]['name'];
		}
	}
	return $hubSpotList;
}

The above function returned an array of HubSpot pages, where each page consists of a title and a link. Of course, we could have retrieved more information, but that’s what our client wanted. Note that the $apiKey parameter should be the HubSpot API Key created in the first step above.

What was really impressive for us is that the above function worked quickly, very quickly. It didn’t delay the page loading at all, but we decided anyway to cache the module’s results for a whole day – just in case (we don’t like surprises that are caused by a sudden under-performance of a 3rd party application).

If you want to grab data from HubSpot and display it on your Joomla website, then the above should be very helpful. If you need more help, then we are ready to help! Just contact us and let us know your requirements, and we will surely implement them for you quickly and for a very affordable fee.

Integrating ListHub with Joomla

Lately, we spent a lot of time working on a project integrating ListHub, the famous MLS platform, on an existing Joomla website. The project consisted of finding “special” listings (those listings with special criteria) within ListHub‘s listings and then displaying them on the site. Despite the fact that it was our third project that consisted of integrating ListHub with Joomla so far this year, we still had quite a few challenges particularly when it came to grabbing the data from ListHub and then parsing that data.

Before delving into the challenges, let us briefly explain what we did in the project:

  • We created a Joomla component called com_realestate (we were going to call it com_listhub, but we then changed our minds because it was possible that our client would then switch to another MLS provider).
  • We created a hash-protected view that was responsible for grabbing the data (e.g. downloading the XML file from ListHub‘s site).

  • We then created another view, which was also hash-protected, that was responsible for parsing the XML file downloaded in the previous step and then storing the selected entries (and information) in the database. (Note that these two views can be merged into one view – but we chose to separate them into two).

  • We created a paginated view that displayed a listing of all the entries that were stored in the database (with sorting options).

  • We created a search plugin that scanned all the entries for the searched text, eventually feeding the search results to the view created in the previous step.

  • We then created a search module which made use of the search plugin above.

  • Finally, we created a view that listed the details of a single real estate entry in the database.

Of course, the project was much, much more than the above but the above information is essentially what we can share with the public…

So, as you can see, the project is pretty much a straightforward MVC Joomla extension, with the exception of the first couple of steps (which are always a challenge). So, in order to make things easier for those who would like to do a similar project, we decided to share the main code for the first two steps!

In order to grab the data the XML file from ListHub, we used the following code (note that we are using cURL, so make sure that you have the cURL PHP module installed on your environment):

$jinput = JFactory::getApplication()->input;
$hash = $jinput->get('hash', '');
if ($hash != '[the_secret_hash]')
	die('Unauthorized...');

error_reporting(E_ALL);
set_time_limit(0);
define('XMLNS_COMMON', 'http://rets.org/xsd/RETSCommons');


//The following 5 variables are typically stored in the extension's settings and not hardcoded into the code.
$url = 'https://feeds.listhub.com/pickup/[ourclient]/[ourclient].xml.gz';
$username = '[ListHub username]';
$password = '[ListHub password]';
$filePath = JPATH_SITE.'/xmldata/listhub.xml.gz';
$xmlFilePath = str_replace('.gz', '', $filePath); 


//Get the gzipped file from ListHub
$downloadedFile = fopen ($filePath, 'w+');
$curlHandler = curl_init();
curl_setopt($curlHandler, CURLOPT_URL, $url);
curl_setopt($curlHandler, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curlHandler, CURLOPT_USERPWD, "$username:$password");
curl_setopt($curlHandler, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($curlHandler, CURLOPT_FILE, $downloadedFile);
$result = curl_exec($curlHandler);
$info = curl_getinfo($curlHandler);
curl_close($curlHandler);
fclose($downloadedFile);

//Now that we have the gzipped file, we will need to extract it.
$bufferSize = 4096;
$gzippedFile = gzopen($filePath, 'rb');
$xmlDownloadFile = fopen($xmlFilePath, 'wb'); 
while(!gzeof($gzippedFile)) {
  fwrite($xmlDownloadFile, gzread($gzippedFile, $bufferSize));
}  
fclose($xmlDownloadFile);
gzclose($gzippedFile);

The above code will download the zipped XML file (from ListHub‘s server) containing all the data that ListHub has made available for a particular account. Once the zipped XML file is downloaded, the last part of the code will extract it and we will have a pure, unzipped XML file containing all the listings.

Now that we have downloaded and unzipped the XML file, then we need to parse it. Typically, parsing an XML file is not that hard, but if you want some buried values from within a complex XML file, then it’ll be hard enough.

Here’s how we parsed the XML file:

//Get the XML data from the XML file
$xmlListHubListings=simplexml_load_file($xmlFilePath) or die("Error: Cannot create object");
$allListHubListings = $xmlListHubListings->Listing;

//Loop through all the XML listings
for ($i = 0; $i < count($allListHubListings); $i++){
	$singleListing = $allListHubListings[$i];
	$listingNumber = addslashes($singleListing->MlsNumber);
	$listPrice = addslashes($singleListing->ListPrice);
	$listingKey = addslashes($singleListing->ListingKey);
	$listingUrl = addslashes($singleListing->ListingURL);
	$bedrooms =  addslashes($singleListing->Bedrooms);
	$bathrooms =  addslashes($singleListing->Bathrooms);
	if (isset($singleListing->Photos) && isset($singleListing->Photos->Photo)){
		$imageUrl = addslashes($singleListing->Photos->Photo[0]->MediaURL);
		$numberPhotos = count($singleListing->Photos->Photo);
	}
	else{
		$imageUrl = '';
		$numberPhotos = 0;
	}
	$address = $singleListing->Address->children(XMLNS_COMMON);
	$addressStreet = addslashes($address->FullStreetAddress);
	$addressCity = addslashes($address->City);
	$addressState = addslashes($address->StateOrProvince);
	$addressPostalCode = addslashes($address->PostalCode);
	$longitude = '';
	$latitude = '';
	if (isset($singleListing->Location)){
		$longitude = addslashes($singleListing->Location->Longitude);
		$latitude =  addslashes($singleListing->Location->Latitude);
	}
}

The above code will loop through the extracted XML file that was created in the previous step. It will loop through all the listings and retrieve the main values in each listing (of course, there are many more values that can be retrieved from the XML file, so it's really up to your requirements). An important note here is that ListHub requires a minimum amount of information to be displayed on the site (for example, they require that you include the name of the Real Estate agent/company, which is not retrieved in the above code).

By the way, if you want to refresh the ListHub data on your website daily, then you will need to include a call to the above views in your cron. That's how we do it anyway... Of course, you can hire a person who will refresh those views every day, but that would be completely inefficient! (OK - that was a joke...)

Now, in this post, we have tried to make things much, much easier for those trying to integrate ListHub with Joomla (we honestly think that the above code, with just one little exception, can be used in any other CMS, including WordPress and Drupal). We hope we helped! In case you need more help or in case the above code did not work for you for one reason or the other, then we're here for you! Just contact us and we'll integrate ListHub for you on your Joomla website quickly, professionally, and for a very affordable fee!

How to Change K2’s Maximum Upload Size

We can never praise K2 enough – we think it handles content even better than Joomla (we did write a post about its advantages). It’s not perfect though, and it does, occasionally, suffer from some inconsistencies.

For example, if you’re editing an item in K2, and you scroll down to the bottom of the editor, and then you click on the Image button, then you’ll notice that the maximum upload size of the image is the same as that of Joomla (which is set at the Media Manager level).

However, if you click on the Image tab on the top, you will notice that the Max upload size is set to the maximum upload limit defined in your global php.ini (for many servers, that maximum upload limit is typically set to 2 MB). If you want proof about this, then open the file default.php located under the administrator/components/com_k2/views/item/tmpl folder and check line 242:

<i>(<?php echo JText::_('K2_MAX_UPLOAD_SIZE'); ?>: <?php echo ini_get('upload_max_filesize'); ?>)</i>

As you can see from the above line, K2 is setting the maximum upload size to the upload_max_filesize configuration setting in the global php.ini.

The above also applies to the maximum upload limit for media files (under the Media tab).

So, how to address this problem?

Addressing this problem is easy if you are on a VPS or on a dedicated server. All you need to do is to create a file called .user.ini with the following content…

upload_max_filesize = "256M"
post_max_size = "256M"

…and then upload it to the root directory of your website.

Once this is done, K2’s maximum upload will increase to a whopping 256 Megabytes! (If you want more than 256 Megabytes, then all you need to is to change the number in the .user.ini file from 256 to a higher number of your choice). If you are on a shared hosting environment, then some companies won’t allow the override of certain PHP settings through .user.ini files – in this case it is a good idea to contact your host and ask them to increase the limit for your website (some hosts will acquiesce to such requests).

We hope that in future iterations, K2 will address the problem and the maximum upload limit (across the board) will be the same as Joomla’s. Meanwhile, you can rely on the solution above to fix the problem (the best thing about it is that you won’t have to modify a single K2 file!).

If you need help implementing the above solution, then all you need to do is to contact us. Our price is right, our work is professional and quick, and we are the friendliest programmers on the Milky Way.

A Script to Evenly Distribute Files Under the Joomla “images” Directory

Many Joomla administrators make the mistake by uploading all their images to the root directory of the images folder. A mistake, as they all soon discover, that can cause the media manager to crash.

At itoctopus, we have come up with a solution to this problem, and we call it: EDIJ (short for Evenly Distributing Images in Joomla – OK, that was a joke!). No seriously, we did come up with a solution a long time ago (and we use it all the time), and the solution is to evenly distribute images under the images directory based on the image’s modification date. Here’s how we do it:

  • We loop through all the images under the root directory of the images folder.
  • We get the last modification date of each image (the best you can get with standard Linux is the last modification date).

  • We extract the year and the month from the modification date of that image.

  • We calculate the new path of the image (which will be something like images/[yyyy]-images/[mm]).

  • We then move (using the PHP function rename) the image from the old location to the new location.

If you want to see it in action, then you can find the script here! Yes – we’re that generous! But, before using it, please keep in mind the following:

  • We don’t claim any responsibility, explicit or implicit, resulting from the use of the script. You use the script at your own risk!
  • This script modifies the filesystem and the database – so you should make sure you backup your Joomla website before running it.

  • You will need to modify the path at the end of the script to correspond with the current physical path of your Joomla website.

  • You will need to ensure that Apache has full permissions over the images directory.

  • Ideally, you should only run the script during off hours or on a staging version of your website.

  • You will need to extract the script to the root directory of your Joomla website, and then run it by pointing your browser to: http://www.yourjoomlawebsite.com/evenlyDistributeImagesJoomla.php.

  • The script will work for Joomla articles and K2 items (by default, it will work for Joomla articles, you will need to point the main function to the K2 function for it to work for K2).

  • The script will take some time, so be patient.

We understand that for non-technical persons, such a script might be a challenge. If that’s the case, then just contact us. We’ll make sure to handle this in as little time as possible and for a very, very affordable fee!

Your Joomla Website Is Crashing After a Dramatic Load Spike? Check the Swap!

Before we begin, let us explain what swap is…

Swap, in server terms, is hard disk space reserved by the system to be used as a backup memory in case the RAM is fully used. Once a server starts using swap memory, then you will notice some degradation in the performance because the RAM is always much, much faster than a hard disk. SSD hard disks fare much better when the system starts using the swap.

A very large racing website (built on Joomla) experienced extremely heavy load in the past weekend. The website eventually crashed. Granted – the website was getting a record number of visitors because of a huge racing event, but the website was highly optimized and it was a bit weird that it crashed. The site’s administrator called us for help, and, of course, we obliged…

When a large website crashes, we take the following steps:

  • We ssh to the site’s server as root.
  • We stop MySQL and then Apache by executing the following commands in the shell:

    /etc/init.d/mysql stop
    /etc/init.d/apache2 stop

  • We add the following to the very beginning of the .htaccess file (which is located under the root directory of the website):

    deny from all
    allow from <Our IP Address>

  • We restart Apache and MySQL using the following commands:

    /etc/init.d/apache2 start
    /etc/init.d/mysql start

Doing the above will ensure 2 things: 1) We have exclusive access to the website, and 2) all the load associated with the website is no more.

So, after doing that, we examined the slow query log and we discovered that there was nothing of importance there. The slow query log was very slim (it was a few hundred kilobytes and it was gathering slow queries for weeks…).

We then checked the disk space available on the hard disk (maybe we ran out of disk space) using the following shell command:

df -m

To our disappointment, the server had more than ample disk space in all its partitions! So that wasn’t a disk space problem.

We then checked the memory allocated to the server to see if there’s something wrong there, using the following command:

free -g

Here’s what we got:

             total       used       free
Mem:            16         12        4
Swap:           4          4         0

Aha! The system ran out of swap! Not only that, for some reason, the swap allocated to the system was sort of a locked! Which meant that the Joomla website wasn’t able to use it when the load gets serious! So what we did was that we released the swap by turning it off, and then we turned it on again by issuing the following 2 shell commands:

swapoff -a
swapon -a

We then removed the deny from all line added to the .htaccess to allow everyone to access the website. Once we did that, we monitored the load and it was OK, but we knew that it’s only a matter of time before the same problem happens again… Why?

Because of 2 things:

  1. If the system runs out of swap, it means that it ran out first of memory, which means that the memory is no longer enough for that large website (the website gets about 2 million visits every month and it is in the top 10k most visited websites in America).
  2. Even if we accept the fact that the system has to touch the swap then and now, the swap allocated is too little. In fact, we think that the amount of swap on a server should be the same as the RAM. So, if you have 16 GB of RAM on your server, you should have 16 GB of swap.

So, how did we solve the problem for the long term?

We solved the problem pretty quickly and easily: 1) We asked the host to upgrade the RAM on the server to 32 GB, and 2) We asked the host to increase the amount of swap to 32 GB. Doing that ensured long term stability of the website.

If your website crashes often immediately after a huge load, then check the swap on your server. If you see that your swap is full, then most likely your server’s memory is no longer enough and you need to increase it (and also increase the swap). If you think this whole thing is not exactly in your court, then throw it in ours, and let us handle the problem. Just contact us and let us prove to you how fast, efficient, professional, and affordable our work is!

“The requested URL index.php was not found on this server.” Error on Joomla

A new international client called us very early in the morning today and told us that he was seeing the following error when trying to access his website:

“The requested URL index.php was not found on this server.”

We immediately checked the problem and we noticed that the error was on every page – so it meant one of the following:

  • The index.php was mistakenly (or maliciously) deleted.
  • The index.php file cannot be read by the webserver because of permission issues (for example, the permissions on the index.php file are “000” [yes, we’ve seen this happen before, but the displayed error was different though]).

  • There is a problem with the .htaccess file.

Naturally, the first thing that we did was that we ssh’d to the server and checked the index.php file. Everything seemed OK: the file was there, its permissions were correct (they were set to “444”), and the content of the (core) file hasn’t been modified.

So we then checked the .htaccess file, and we didn’t find anything wrong with it. It seemed like the default .htacess with no modifications whatsoever.

Hmmm…

Our next approach was to check whether the index.php file is being loaded and so we added the following line to its beginning (immediately after the opening PHP tag):

die('Under maintenance...');

We then tried to browse to the website but we got the same error. This of course means that Apache is not even loading (or trying to load) the index.php file.

So we started thinking, what could have changed from a few hours ago (the client told us that the problem happened a couple of hours before calling us)? To answer our question, we ran the following command in the shell:

find /home/[user]/public_html -path '/home/[user]/public_html/cache/*' -prune -o -mtime -1 -ls

(The above command will list all the files that were changed in the past 24 hours)

And we got the following:

1573337 152 -rw-r–r– 1 [user] [user] 2859 Apr 30 07:22 /home/[user]/public_html/.htaccess

Aha! So the .htaccess file was modified! Naturally, we took another, thorougher, look at the .htaccess file and we noticed that this line:

# RewriteBase /

has been changed to this:

RewriteBase /

(The sharp is missing from the second line, which means that the line was uncommented.)

And so we re-added the sharp sign (or hash, or the number sign, or whatever people are calling it these days), and then we checked the website, and guess what? It was working! Woohoo! The client later told us that he did make this modification to address another problem, but he didn’t think that it might cause this kind of destruction!

So, the next time you see that error on your website, a good idea would be to check for these 3 things: the presence of the index.php file, the permissions on the index.php file, and the presence of the sharp (#) sign next to RewriteBase / in your .htaccess file. If you still can’t find the problem, then go ahead and contact us. We will solve your problem in as little time as possible and you will only have to pay our super affordable fees!

How to Disable Email Cloaking in Joomla on a Particular Page

We’re not the biggest fans of Joomla’s Email Cloaking plugin; we think it’s useless. But, obviously, the Joomla development team doesn’t agree with us as the plugin still exists in Joomla 3.4.x (the latest version of Joomla at time of writing this post). Additionally, there are still many Joomla websites who still use this plugin because they (mistakenly) believe it will protect their emails from spam. But, at times, some of these websites start having issues on some pages, and so their administrators reluctantly decide to completely disable the Email Cloaking plugin, thinking that this is the only way to address the problem.

While we think that disabling the plugin is the best way to address any problem caused by it (we really, really don’t like it), we have to admit, it is not the only way. In fact, one can disable the plugin at the problematic page level by just adding the following code to the (beginning of the) content item having the issue (the content item can be a Joomla article, a K2 item, a Joomla custom HTML module, etc…):

{emailcloak=off}

Once the above code is added to a content item, then the plugin will not be applied on that content item. This is because, in its code, the plugin has the following lines:

if (JString::strpos($text, '{emailcloak=off}') !== false){
$text = JString::str_ireplace('{emailcloak=off}', '', $text);
return true;
}

As you can see from the above code, the plugin is instructed not to continue running in case it encounters {emailcloak=off} in the content item.

If you would like to disable Joomla’s Email Cloaking plugin for a certain article (or any other content item), then all you need to is to follow the above code. If it doesn’t work for you, then just contact us and we’ll do it for you in no time and for a very affordable fee!

Internal Server Error When Trying to Save Content on a Joomla Website

One of our clients called us today and told us that they were trying to save an Event in the DPCalendar extension, and they were getting an Internal Server Error. They said that the same problem was happening when they were trying to save a blog in EasyBlog.

From our experience, the majority of Internal Server Errors are caused by Apache’s mainstream security module, ModSecurity (an example of this can be found here).

So, the first thing that we did was that we checked the ModSecurity log file (which is the modsec_audit.log file located under the /usr/local/apache/logs folder) and we found the following errors:

Message: Access denied with code 500 (phase 2). Pattern match “(insert[[:space:]]+into.+values|select.*from.+[a-z|A-Z|0-9]|select.+from|bulk[[:space:]]+insert|union.+select|convert.+\\(.*from)” at ARGS:jform[description]. [file “/usr/local/apache/conf/modsec2.user.conf”] [line “371”] [id “300016“] [rev “2”] [msg “Generic SQL injection protection”] [severity “CRITICAL”]
Message: Rule 10ab6e8 [id “3000086“][file “/usr/local/apache/conf/modsec2/rootkits.conf”][line “155”] – Execution error – PCRE limits exceeded (-8): (null).

Aha! So, it was it was ModSecurity (Tweety?). We had 2 options, we either had to disable ModSecurity for everyone (we can’t just disable it for our client since they had a dynamic IP), or we had to address the problem differently. We chose the latter.

In order to solve the problem, we created a file called whitelist.conf under the /usr/local/apache/conf/modsec2 folder with the following content:

<LocationMatch "/administrator/index.php">
  SecRuleRemoveById 300016 3000086
</LocationMatch>

(Note that the numbers in red are the same as above.)

We saved the file and restarted Apache, and the problem was solved!

But why was the problem solved?

It’s because in the file whiltelist.conf, we instructed ModSecurity not to enforce rules 300016 and 3000086, which were causing the problem.

But what about disabling ModSecurity altogether? Wouldn’t that solve the problem?

Yes – it would, but doing so will have some severe adverse effects when it comes to the website’s (and the server’s) security. We don’t really recommend doing this as ModSecurity is extremely efficient for filtering out malicious requests to the server. It is possible to disable ModSecurity at the .htaccess level or the (virtual) server level if you have a VPS or a dedicated server. However, if you are using a shared hosting solution, it might be that your host does not allow the disabling of the ModSecurity module. In the latter case, you have no other option but to try to talk to your host to make an exception for your website.

If you are running into Internal Server Errors when you are trying to add/edit something on your website, then most likely this is a ModSecurity issue. Just follow the above guide and you should be OK. If it doesn’t work for you or if you feel it’s a bit complicated, then please contact us – we’ll help you address the problem in no time and for a very affordable fee!

Why AcyMailing Causes Load Issues on the Server

A huge racing website that we manage sends out daily newsletters using AcyMailing (the famous newsletter extension for Joomla), and every time those newsletters are sent, the website slows down gradually and eventually crawls to a halt. The whole performance degradation time is typically only a few minutes, which has always been tolerable by the website administrators – until Sunday (a couple of days ago), when they had several newsletters to send because of a huge racing event. They just couldn’t tolerate 30 minutes of website unresponsiveness and so they asked us to investigate and fix the problem prior to sending out those newsletters.

Typically, we don’t recommend our large clients to use Joomla extensions that use the application’s mail server to send out newsletters, not because we think they’re unreliable (on the contrary – we think they are very reliable), but it’s because we think (for many reasons – including load issues, blacklisting, server management, etc…) that the server hosting the main website should not be used as a newsletter server. What we do recommend is using a 3rd party tool to send out emails (most of these tools have extensions which allow them to be used from within Joomla, but without using the Joomla server as a mail server), for example ConstantContact or MailChimp. But some clients send out many newsletters to many people every day, which makes using a 3rd party tool a non-viable alternative from a financial perspective.

Going back to our original client’s request, we started investigating the problem by checking the MySQL slow query log, and it didn’t take us long to discover the root cause: it was a myriad of slow INSERT queries to different tables in the eximstats database. Every time AcyMailing sent an email to one individual, several INSERT query statement were executed on different tables of the eximstats database. Since each table was fairly large, some of these statements were taking several seconds to execute.

Since our client was relying on AcyMailing‘s statistics (and not on Exim‘s), we decided to disable statistics on Exim (Exim is one of the most used mail servers out there). Here’s how we did it:

  • We logged in to WHM on the server hosting the Joomla website.
  • We clicked on Service Configuration.

  • We then clicked on Service Manager.

  • We unchecked the box next to Eximstats.

  • We clicked on Save at the bottom (you will have to wait for about half a minute when you click on that Save button).

  • We restarted the Exim mail server by going back to the Home page on WHM, and then clicking on Restart Services, and then clicking on Mail Server (Exim), and finally clicking on Yes.

Doing the above ensured that Exim no longer collected statistics about the sent emails, which resolved the problem as the website stopped having load issues during newsletter sending time.

If you are using AcyMailing and you are having load issues (or even downtime) during the sending process, then try disabling Exim‘s statistics. If that doesn’t fix the problem, then a good idea would be to contact us. Our fees are right, our work is super professional, we love what we do, and we are the friendliest developers on Planet Earth.

Updating the Hits Counter on Your Joomla Website Taking a Long Time? Read This!

A client called us back on Friday and told us that his company was having some serious load issues on its high traffic corporate website (which was, or course, using a Joomla website). Usually, when there are performance issues on a website, the first thing that we check is the slow query log (typically located under the /var/lib/mysql directory). And so we did, and we noticed that the log was full of queries that are more or less the same as the ones below:

UPDATE #__categories SET hits=hits+1 WHERE id = 402;
UPDATE #__content SET hits=hits+1 WHERE id = 114317;

Each query was taking a few seconds (some of these queries were taking about 10 seconds) despite the fact that it updates only one row. Hmmm – but why?

We tried running one of the above queries directly in phpMyAdmin and it took more or less the same amount of seconds every time we ran it. Oh, and by the way, the website was using MyISAM as then engine for the MySQL database, which means that any update to the table (such as #_content) will lock the whole table, which means that any subsequent update to the table will have to wait on the previous update to complete before getting executed. Of course, switching to InnoDB will solve the table locking issue (since InnoDB uses row locking instead of table locking, which means that an update to a specific row will only lock that row), but may introduce a myriad of other issues, and we still won’t know what is causing the update to take a long time in the first place.

And so, we decided to stick with MyISAM and solve the problem. So, the next thing that we did was checking the MySQL configuration, which can be found in the my.cnf file which is located under the /etc folder. Luckily we immediately discovered the problem, and it was in the following lines:

query_cache_size=8G
query_cache_limit=16M

The database had a query cache of 8 GB (that’s eight gigabytes!), and, while cache is a good thing (or maybe not), it can cause some serious issues when it is above a certain limit (that limit depends on the processing power of the server, its memory, as well as other factors), because the overhead of maintaining the cache will increase substantially. So, on the website we were working on, everytime there was an update to the #__content table (which is one of the most used tables on a Joomla website) because of hit tracking, all the queries using the table #__content got invalidated, which meant that MySQL had to scan through gigabytes of data for those queries and remove them. Not only that, all these invalidated queries must be then re-executed, which created even additional load on the server. Yes – it’s a very, very vicious circle.

So, what was the solution?

The solution was to significantly reduce the query cache by replacing the above lines with the following lines (the last line will reduce the size of a cache block, which is a good idea to do if most of the cached queries are less than 2 KB in size, which is typically the case on a Joomla website) and then restarting MySQL (the below changes will not take effect unless MySQL is restarted):

query_cache_size=128M
query_cache_limit=256K
query_cache_min_res_unit=2k

By doing the above, we ensured that the cost of maintaining the query cache will not be that significant, which meant that update queries to the most used tables will no longer be heavy queries, which resulted in a much, much better load on the server (the load got reduced from an erratic 12-20 to a consistent 2-3).

Another solution, of course, was to completely disable hit tracking on the Joomla website, which would have solved the problem probably more efficiently. However, it wasn’t an option for us since the website was displaying the number of hits on each article.

If you’re having performance issues on your website because of hit tracking, then consider disabling it if you’re not using it. If you are, then you should check the configuration parameters of the MySQL database engine on your server, and ensure that you are using reasonable values for the query cache parameters. If you need help, then please contact us and we’ll do the work for you in no time and for a very reasonable fee!

How to Display a Joomla Module Only on Mobile Devices

We are getting an increasing number of clients telling us that they want to display certain modules only for mobile devices (and some other modules only for computers). This is very normal, since all websites are seeing a dramatic increase in mobile traffic, a fact that is causing businesses to target mobile visitors with some different content (such as different advertisements) in order to increase conversions.

In this post, we will explain exactly how we do this in a very clean way. Without further delay, let’s start!

  • We first open the file index.php located under the templates/[template-name] folder and we add the following function immediately after the defined( ‘_JEXEC’ ) or die( ‘Restricted access’ ); line:

    function isMobileTraffic(){
    	$arrMobileAgents = array('android', 'blackberry',
    	'googlebot-mobile', 'ipad', 'iphone', 'ipod', 'mobi',
    	'mobile', 'opera mini', 'safari mobile', 'windows mobile');
    	$strCurrentUserAgent = $_SERVER['HTTP_USER_AGENT'];
        foreach ($arrMobileAgents as $strMobileAgent) {
    		if (stripos ($strCurrentUserAgent, $strMobileAgent) !== FALSE) {
    			return true;
    		}
    	}
    	return false;
    }

    The job of the above function is to detect whether the traffic is coming from a mobile device. If it is, then the function will return true. If not, then it will return false. Please note that the above function is not exhaustive, so it might be that there are some rare user agents that are not taken into consideration (in that case, the function will treat traffic coming from these devices as desktop [computer] traffic).

  • After adding the above function, we add the mobile exclusive position to the template; we just locate the position where we want to add the module and we add the following code:

    <?php if (isMobileTraffic()) { ?>
    <div id="mobile-module-1">
    	<jdoc:include type="modules" name="mobile-module-1" style="raw" />
    </div>
    <?php } ?>

  • Once the above is done, we create a module (it can be of any type), and then we assign it to the mobile-module-1 position. Once that is done, the website will display that module only for mobile traffic! Easy, huh?

But what if we already have a template that is dedicated for mobile traffic?

In this case, you only need to create a unique position (e.g. a position that doesn’t exist in the non-mobile template) in the index.php of the mobile template, and you’re all set! (you don’t have to add the function above to the beginning of the index.php file.)

We have tried to make this guide as clear as possible. But, it is impossible for us to make it code free. If you’re not a programmer and you need help implementing the above, then all you need to do is to contact us. Our fees are super affordable, our work is super fast, and we are super professional!

“MySQL server has gone away” Error on Joomla

One of our regular clients was seeing the following error when trying to view the VirtueMart orders in the backend of his Joomla 3.3.6 website:

Error displaying the error page: Could not connect to MySQL.: MySQL server has gone away SQL=SELECT SQL_CALC_FOUND_ROWS o.*, CONCAT_WS(‘ ‘,u.first_name,u.middle_name,u.last_name) AS order_name ,u.email as order_email,pm.payment_name AS payment_method FROM #__virtuemart_orders as o LEFT JOIN #__virtuemart_order_userinfos as u ON u.virtuemart_order_id = o.virtuemart_order_id AND u.address_type=”BT” LEFT JOIN #__virtuemart_paymentmethods_en_gb as pm ON o.virtuemart_paymentmethod_id = pm.virtuemart_paymentmethod_id ORDER BY created_on DESC LIMIT 0, 100

Obviously, the above SQL query was causing the MySQL server to go away – but what does going away mean in this context? It typically means that the server timed out while processing the above query (which takes a lot of time).

Hmmm! Easily fixable, we thought. We ssh’d to the server and we edited the my.cnf file (this file contains all of MySQL‘s settings) located under the /etc folder the following way:

  • We opened the file in vi:

    vi /etc/my.cnf

  • We changed the value of max_allowed_packet to 256M.

  • We closed the file and then we restarted MySQL by issuing the following command (this step is necessary so that MySQL starts with the new settings – note that on some Linux installations, the command to restart MySQL is different):

    /etc/init.d/mysql restart

  • We checked the problematic page (the one displaying the VirtueMart orders), and it still had the same problem.

No problem! We opened again the file my.cnf and we added the following lines:

wait_timeout=100
interactive_timeout=100
net_write_timeout=100

We restarted MySQL and tried again, but still, the same problem.

We started to feel a bit of pressure – but we thought – maybe we will be able to find something in MySQL‘s error log. So we checked that error log, and we noticed that there was a problem with the innodb_index_stat table (this is a core MySQL table and it is used for InnoDB tables). We checked that table through phpMyAdmin (it is located in the core mysql database), and we got the following error: table ‘mysql.innodb_index_stats’ doesn’t exist

Huh? At this point, we were no longer surprised by the error that our client was getting, but we were astonished that the rest of the website was working since a core MySQL table, the ‘mysql.innodb_index_stats’ table, was corrupt.

We tried everything – we tried every imaginable method to repair the table – but none worked. Eventually what we did was that we created a backup of the database and then re-installed MySQL, and finally re-imported the database. Everything worked as normal once we did that!

But what caused this problem in the first place?

We’re not certain – we suspect a server crash that happened recently caused this problem. But, in all fairness, the actual reason might not be it. The good news, however, is that we got this problem solved!

If you’re seeing the “MySQL server has gone away” error on your Joomla website, then try the above solutions in sequence. If none of them worked, then your best bet would be to re-install MySQL as quite possibly you have a corrupt instance of the DB server. If you need help fixing this problem then please contact us. Our fees are super affordable, our work is quick and very professional, and we are the friendliest developers on planet Earth.

Calculations Not Working Correctly on RSForm Pro – How to Fix

One of the extremely convenient (and very cool) features in RSForm Pro is the Calculations feature. The Calculations feature can be found by logging to the backend of the Joomla website, and then by going to RS Form Pro -> Manage Forms, and then clicking on the form you wish to add the calculations to, and then clicking on the Properties tab on the top, and finally clicking on the Calculations tab at the bottom left.

Now, the obvious question would be, “what’s so cool about this feature?”

Well, let’s assume you have the following 3 fields on your form:

  • Price
  • Quantity
  • Subtotal

Typically, you would like to have the Subtotal automatically calculated once the person enters/changes the Price and the Quantity. If you were to do this using pure JavaScript, it will take you a lot of time and you will have a lot of code to maintain. But, since you’re using RSForm Pro, you can do this instantly the following way:

  • You go to the Calcuations page as described above.
  • You choose Subtotal from the dropdown.

  • You paste the following formula into the text field just left of Save this calculation:

    Subtotal = {Price:value} * {Quantity:value}

  • You save the form by clicking on the Save button on the top left.

  • You check the form on your website and you watch the coolness in action!

The nice thing about this is that this feature can be used in complex calculations, and, more importantly, in derived calculations. By derived calculations we mean calculations using other calculations. For example, in the above form, you might have 3 subtotal fields: Subtotal A, Subtotal B, and Subtotal C, and each of these fields is the result of Price [n] x Quantity [n]. In this scenario, you may want a field called Total which is the sum of all the subtotals. You can easily do this using the Calculations feature by just assigning the following formula to the Total field:

{Subtotal A:value} + {Subtotal B:value} + {Subtotal C:value}

That’s it! Now when a person changes the price or the quantity of any product, the subtotals and the total will be automatically updated. This is also more convenient and much clearer than the following (longer) formula (which yields the same result):

({Price A:value} * {Quantity A:value}) + ({Price B:value} * {Quantity B:value}) + ({Price C:value} * {Quantity C:value})

But – nothing is perfect and every piece of software has bugs! In this case, the bug is evident when one of the fields is anything but a regular text or hidden field… Let’s say, for example, you have a dropdown field where the person chooses the quantity. In this case, the Subtotal calculation will not work, and consequently, the Total calculation will not work. Why?

Well, because when the field is a dropdown (also referred to as a select), a radio button group, or a checkbox group, the system assumes that the user can choose multiple values (of course, this is not possible in the case of a radio button group or a single select), and so it stores the value in an array, which then causes a problem when the system tries to use this value…

But, how to fix the problem?

There are several ways to fix the problem, and the easiest one, in our opinion, is to fix it in the script.js file, which is located under the components/com_rsform/assets/js folder. Here’s how you can fix the problem for the dropdown:

  • Open the aforementioned file.
  • Search for the line:

    values.push(element.options[o].value);

  • and add the following line before the above line:

    return element.options[o].value;

  • Save the file, clear the cache, reload the page, and the problem should be solved!

You can apply the same fix for the radio buttons and the checkboxes! It is a very simple fix. The downside of this solution is that if you have a multiple select field that will be used for calculations in your form, then only the first value will be taken into consideration!

Now, if you, our very precious reader, are facing the same problem and if the above guide didn’t solve the problem (or if you feel that the whole thing is a bit complicated and you prefer that someone else does it for you), then please contact us, we will do it for you swiftly, professionally, and for a very affordable fee!

“Fatal error: Call to undefined method stdClass::render()” when Using JComments

One of our clients was getting the following error after upgrading Joomla from 3.3.6 to 3.4:

Fatal error: Call to undefined method stdClass::render() in /components/com_jcomments/libraries/joomlatune/template.php on line 318

He told us that immediately after he upgraded to Joomla 3.4, JComments (the famous Joomla commenting platform), stopped working (clicking on the Add Comments link stopped working). So, what our client did is that he uninstalled and then re-installed JComments – but when he did that, he got the above error, and so he called us!

Luckily, the above error told us clearly where the problem was. So, we opened the file template.php located under the /components/com_jcomments/libraries/joomlatune/ folder, and we jumped to line 318, and here’s what we saw:

$this->_templates[$template]->render();

Hmmm… So the template cannot be rendered, but why? So, we went to the JComments settings in the backend, by clicking on Components -> JComments -> Settings, and then clicking on the Layout. We then quickly checked the Template field (in the View section) and we immediately noticed that no template was assigned. Naturally, our first instinct was to assign a template to the Template field, and so we did that (we chose the default template) and we checked the website, and guess what, the problem was no more!

So, what did cause the problem in the first place?

The problem was caused by the fact that our client was using a custom template that did not exist in JComments public distribution, and so when he uninstalled JComments, that template was removed. The new installation of JComments used the old settings, and so it set the Template field to the non-existing template – hence the problem.

If you’re seeing the same problem on your Joomla website and if you are using JComments, then most likely the problem is that there is no template assigned in your JComments settings. If you already have a template assigned, then make sure that all the files pertaining to that template exist. If you are also sure about that, then your best option would be to contact us. We will fix the problem for you in as little time as possible and for a very affordable fee!

A Quick Way to Optimize a Joomla Website: Logging MySQL Queries that Lack Indexes

If you have already optimized many of the slow queries on your Joomla website using the power of the MySQL slow query log, and yet MySQL is still putting some pressure on your server load, then a good idea would be to check which indexless queries are being executed by your database server. By indexless queries, we mean those queries that have filters or sorting, but the fields that these queries are filtered or sorted by are not indexed.

For example, if you have the following query somewhere on your Joomla website:

SELECT * FROM #__content WHERE cat_id=5 ORDER BY created DESC";

Then you should have both cat_id and created indexed.

Now the question is, how do you know easily which queries lack indexing? Well, luckily, there is an easy way to do it in MySQL if you have root access to the server.

All you need to do is the following:

  • ssh to the server hosting the database of your Joomla website.
  • Open the file my.cnf located under the /etc folder (that file might be located elsewhere, but on all the servers we have worked on, it was located under the /etc folder).

  • Add the following lines at the end of the file:

    slow-query-log=1
    long-query-time=1
    log_queries_not_using_indexes=1
    slow-query-log-file="/var/lib/mysql/slow.log"

  • Save the file (if you’re using vi you can quickly save the file by clicking on ESC and then typing :wq! followed by the ENTER key).

  • Restart MySQL using the following command:

    /etc/init.d/mysql restart

  • Now keep an eye on the slow.log file located under the /var/lib directory, you will notice that it will get filled with all sorts of queries, quickly!

Once you have enough queries then you should disable indexless query logging the following way:

  • Open the file my.cnf under the /etc folder.
  • Comment out the following line by adding a sharp (or a hash – depending on what you call it) to its beginning:

    log_queries_not_using_indexes=1

    (so the above line will be #log_queries_not_using_indexes=1)

  • Save the my.cnf file and restart MySQL.

Now you can examine all the indexless queries in the slow.log file and fix them. You should focus first on the queries that you see most in the slow.log file. Once you fix those queries, repeat the process above (re-enable/re-disable the logging of the queries that have no indexes) until all (or most) of these queries are fully optimized. Now check your load, and you should see a substantial difference.

Note that you will need to have some serious database skills to fix those queries. If you need help, then you can always contact us. Our rates are super affordable, our work is very clean and professional, and we are courteous and super fun to work with!

How to Completely Get Rid of the “Invalid Token” Error on Joomla

Note: The solution in this post is compatible with all versions of Joomla that are 2.5 and above (as of March 2015).

Another note: This solution will always work on compatible Joomla versions.

Warning: The solution in this post is a core modification – this means that your changes will be possibly wiped out with a future Joomla update.

Another warning: Go with the solution presented in this post only when you’re desperate – since this particular core modification may jeopardize the stability and security of your Joomla website. You have been warned!

Final warning: This solution should be temporary if you decide to implement it. You should locate the root cause of the “Invalid Token” issue and fix it there. (if you need help with that, we’re always there!)

Ah – invalid tokens – if we had a dime everytime a client calls us with an invalid token issue, we’d have a small jar full of dimes! Invalid tokens are by far the most annoying, frustrating, and loathed Joomla errors and they’re nearly impossible to fix. But, what most people do not know, is that one can get rid of these errors once and for all with just one line of code! Here’s how to do this:

  • Open the file session.php located under the libraries/joomla/session folder.
  • Locate this line:

    public static function checkToken($method = 'post')

  • Just after the opening bracket (the {), add the following line:

    return true;

  • Upload the file session.php back to your website.

  • That’s it! You have solved the “Invalid Token” error on your Joomla website once and for all (well, at least until the next update of your Joomla website)!

Unbelievable, huh? Just one line of code in the right place marked the end of many, many hours (or many weeks, for some) of frustration.

But, doesn’t this solution seem like removing one small part of a car and claiming that it’s not needed?

Yes – that’s exactly what this solution is – but we’re not car repairmen and we’re not claiming that this functionality is not needed – it is needed! In fact, it prevents people from running automated queries in order to gain access to your website by directly submitting content to your forms instead of going through the normal process – and that is exactly why we wrote a warning at the beginning of this post that this should be only a temporary solution until a real fix is implemented.

So, will this solution really jeopardize the security of my Joomla website?

It might. As stated above, people won’t have to go through the normal process to submit actions to your website, which means that people may 1) login to your website using a brute force attack or 2) spam your website. Note that if you have a very strong password then the former possibility is a bit remote.

If you need help implementing this solution, or if you need help identifying and fixing the real problem, then please contact us. We can do the work for you in very little time and for a very affordable fee!

The MySQL Error Log Is an Excellent Tool for Debugging a Joomla Website

A couple of days ago a client called us and told us that whenever someone tries to click on a link in an ACYMailing newsletter, he was seeing a blank page. The client assured us that he didn’t change anything in the code.

If you have been reading our blog for a while, then you will probably know that a blank page is a sign of an error, and that in order to see that error, you will need to set the Error Reporting level to “Maximum” in your global configuration settings. And so we did that, but we still didn’t see any error. Easily fixable, we thought! But despite using every imaginable way in our guide to display the errors, we still couldn’t see what the error was.

But then, for some reason, we thought about checking the MySQL error log – maybe it can tell us something. And so we ssh’d to the server and we went to the /var/lib/mysql directory, and we opened the error file (the error file is usually called *.err where * is typically the name of the server, appended to the main domain name on that server; an example of an error’s file name is: server1.yourjoomlawebsite.com.err ). Here’s what we saw:

150303 9:16:45 [ERROR] /usr/sbin/mysqld: Table ‘./[dbname]/jos_acymailing_userstats’ is marked as crashed and should be repaired

Ah ha! So the problem was with a corrupt table. So, in order to fix the problem, we did the following:

  • We logged in to phpMyAdmin (from the WHM interface).
  • We selected the database serving the Joomla website.

  • We clicked on the SQL tab at the top.

  • We pasted the below SQL query in the textarea:

    REPAIR TABLE `jos_acymailing_userstats`;

  • We clicked on GO at the bottom and we waited for a few minutes until the table was repaired.

  • That fixed the problem!

But, what caused the problem in the first place?

The jos_acymailing_userstats table was a heavily active MyISAM table with substantial number of writes per second. Tables that active (write wise) are vulnerable to crashes if a server hiccup occurs during a write activity (for example, during an INSERT or an UPDATE query). A good idea would be to flip such tables from MyISAM to InnoDB (a performance hit may occur if you do this though; you will need to make some modifications to your my.cnf file if this happens).

As you can see, the MySQL error log was crucial in identifying the problem. So, the next time you see a blank page on your Joomla website, take a glimpse at your MySQL error log and see if you can find anything there. If, however, you feel that this whole thing is a bit over your head, then please contact us. We will be able to fix the problem for you in as little time as possible and for a very, very affordable fee!

Can’t Login to Joomla’s Backend when SSL Is Enabled

A new client emailed us yesterday and told us that every time he tries to login to the backend of his Joomla website, he’s redirected to the login page with no errors (our client called this issue “the admin login loop”). So the first thing that we asked him is whether he checked our post about the same problem. He said that he already did, but he still wasn’t able to resolve the problem.

So we tried fixing the problem using the guide in our post above, but we were unsuccessful. But then we thought, that website uses SSL in the backend, so what if we disable SSL and try to login again? And so we did: we disabled SSL by editing the configuration.php file (which is located under the root directory of the Joomla website) and changed the line:

public $force_ssl = '1';

to:

public $force_ssl = '0';

We then tried to login and (surprise, surprise) it worked! Aha! So the problem was related to the SSL setting in Joomla’s configuration… Maybe it’s a bug in the version of Joomla used by our client? Or maybe it’s a rogue extension causing this mess? We weren’t sure, but we just didn’t have the time to investigate further since our client was on a very tight budget. So we fixed the problem in a clean way using the power of the .htaccess file… Here’s what we did:

  • We opened the .htaccess file which is located under the root directory of our client’s website.
  • We added the following lines immediately after RewriteEngine On:

    RewriteCond %{HTTPS} !=on
    RewriteRule ^(administrator/.*)$ https://www.ourclientjoomlawebsite.com/$1 [R=301,L]

  • We saved the file and we uploaded it back.

  • That fixed the problem while ensuring that the backend still works in SSL mode!

If you have the same problem, then all you need to do is to disable SSL (by setting the $force_ssl variable to 0 in the configuration.php file) and follow the above guide. That’s it!

If, however, you want to solve it from within Joomla (and know what’s really causing this problem), or if you don’t feel at ease editing the configuration.php and the .htaccess files, then please contact us. We are always eager for new work, we are experts in Joomla, our clients love us, and we don’t charge much!

You Can’t Create a Menu Item Alias Pointing to a Menu Item of Type “External URL”

A new client called us and told us that a newly created menu of his, which is created out of another menu, does not work. He said that none of the links work in the new menu, despite the fact that they work in the other menu.

So we looked into the issue and we immediately found the cause of the problem, each one of the menu items in the new menu was pointing to a menu item of type “External URL”. That’s the cause of the problem!

If you have a menu item of type “External URL”, you can’t create a menu item alias pointing to it (the same way you can’t create a menu item alias pointing to another menu item alias). You will need to create another menu item of type “External URL” and point directly to that URL.

But why can’t Joomla automatically point the menu item alias to the external URL?

The short answer is that it’s a bug in Joomla (and this is explained in details in the above link). The problem though is that if you ever decide to change one of those external URLs, you will have to change it in each menu item pointing to it, and not in just one menu item. We know, this is annoying, and we hope that Joomla will get this sorted out in the future. But for now, your only option is to just duplicate the menu item and assign it to the other menu that you wish to use it in.

If you have the same problem, and if our post above didn’t help solve your problem, then try clearing your Joomla cache. If this is still an issue, then please contact us and we’ll solve the problem for you in no time and for a very, very affordable fee!

The Right Way to Handle 404’s in Joomla

404: it’s probably the most known number in the Internet world, and it means that the page is not found. Ironically, 200, the response code that one will get when the page is found and everything’s OK, is not nearly as known. This, from our psychological perspective, sheds a light on the pessimistic nature of the Internet (half-full vs half-empty glass, or sort of…). But hey, we’re not psychologists, so let’s focus on what we know best: Joomla!

By default, when a page does not exist, Joomla displays a generic page with a red 404 – Not found header. Of course, if you are running a professional website, you need to change this page to something more elegant (e.g. making it use the same template as your website), and Joomla’s official documentation recommends you do so by following the below steps:

  1. Create an article called 404 in Joomla’s Article Manager. This article will contain your custom 404 error message. Note that the article can be called anything, but it’s better to call it 404.
  2. Create a menu item, also called 404 (again, it can be called anything – but calling it 404 makes it clear that this is your 404 menu item), which points to your 404 page. This means that going to http://www.yourjoomlawebsite.com/404.html will display your error page (assuming you have SEF turned on, and you’re using mod_rewrite, and you’re adding the .html suffix to your URLs).

  3. Copy the file error.php from the templates/system folder to the templates/[your-template-name] folder.

  4. Add the following code to the beginning of the copied error.php file (after restricted access):

    if (($this->error->getCode()) == '404') {
    	header('Location: http://www.yourjoomlawebsite.com/404.html');
    	exit;
    }

  5. That’s it!

Now, if you go to a page that doesn’t exist on your website, you will be automatically redirected to your fancy 404 page. We know, this is fascinating, but there is a problem…

You see, the above code redirects your non-existent pages to an existent, 404 page, using a 302 redirect. So, if you’re using Google Webmaster tools, non-existent pages will no longer show up in the Not Found tab on the Crawl Errors page, since you are telling Google (and all other search engines), that any non-existent page is temporarily redirected to your 404 page. This means that you won’t be able to fix all the problematic links to your content because you won’t be able to see them!

So, what is the right way to handle 404’s in Joomla?

The right way to do handle 404’s in Joomla would be to add the below code in the copied error.php file instead of the code above:

if (($this->error->getCode()) == '404') {
	header('HTTP/1.0 404 Not Found');
	$errorPageContent = file_get_contents('http://www.yourjoomlawebsite.com/404.html');
	echo $errorPageContent;
	exit;
}

The above code will not redirect to your 404 page, but instead, it will grab the content from your 404 page after telling Google (and, of course, other search engines) that the page is not found. This way, you will ensure that Google’s Webmaster Tools will still list all your non-found pages even after you change to the dull Joomla default 404 page to your new and improved 404 page!

We hope you found this post useful. If you need help with the implementation, then please contact us. We will do it for you in a heartbeat and for a really, really affordable fee!

“No Input File Specified” Error on Joomla 3.3.6

A client called us around 1:30 AM this morning, and told us that his website went down all of a sudden. We checked the website and here’s the error that we saw:

No Input File Specified

Hmmm…

It was probably the first time that we saw this error and so we started by debugging the index.php file under the root directory of the website, by adding a die(); at the beginning, but our code didn’t execute and we still saw the same error. Usually, there are 4 reasons why an index.php change isn’t reflected on the actual website:

  1. Apache doesn’t have write permissions on the index.php file. We checked the permissions and they were owned by the suPHP user and they were 644. So, nothing was wrong there!
  2. The website is showing a cached version of the index.php file. We added the debug code at the very beginning of the file, so caching shouldn’t be even triggered. We also wiped out the contents of the cache folder just for good measure. Still, the same problem!

  3. We are working on the wrong file. We checked, double checked, and triple checked that it was the correct index.php file. So, we were confident that we were working on the right file.

  4. There is a quirk in the .htaccess file. We renamed the .htaccess file to .htaccess.old and we moved it outside the website’s directory (some environments will use the renamed file unless they can no longer access it). Nothing has changed! The “No Input File Specified” error was still there.

We then re-checked the index.php file, as this problem might happen if there is a space somewhere before the opening <?php tag or after the ending ?> tag, but nothing was there. We even created another index.php and moved the old one elsewhere. This didn’t fix the problem, but it helped us identify the problem: While moving the file outside the public_html directory, we noticed that the directory had the following permissions:

drwxr-x--- 27 755 nobody 4096 Feb 13 1:20 public_html/

Hmmm… This means that the public_html directory had a permission of 0700 (while being owned by the nobody user). This in turn means that Apache can’t even read the index.php file because it just can’t read anything in the parent directory.

In order to solve the problem, we changed the permissions on the public_html directory by issuing the following commands while in the Linux shell:

chmod 755 /home/[user]/public_html
chown -R [user] public_html
chgrp -R [user] public_html

Note: [user] should be your cPanel username for that website (without the brackets).

That fixed the problem!

But what caused the problem?

At first we didn’t know, but then our client revealed to us that it was his host that caused the problem, while trying to solve another problem!

If you have the same problem on your website, then check the permissions on both the index.php file and the public_html directory. The problem most likely lies there. If it isn’t, or if you’re not a big fan of touching the permissions, then please contact us. We will fix the problem for you in no time and for a very, very affordable fee!

How to Quickly Remove All Articles in Joomla

In this day and age, everyone wants more content on their website (after all, content is king), so for many, this post won’t make any sense. But for some, it does. In fact, we had some clients who actually wanted to reset the content of their Joomla website, despite the potential SEO implications (which we communicate to our clients), but, when the client insists against our best judgement, we will eventually oblige and we delete all the articles off his Joomla website. Here’s how we do it:

  • We login to phpMyAdmin (usually from within cPanel), and then we select the database that powers the Joomla website.
  • We run the following queries (by clicking on the SQL tab on the top, pasting the queries, and then clicking on Go):

    DELETE FROM `#__content`;
    DELETE FROM `#__categories` WHERE `type` = 'com_content';
    DELETE FROM `#__assets` WHERE name LIKE 'com_content.category.%' OR name LIKE 'com_content.article.%';

    Note: You must replace #__ in the above queries to the database prefix of your Joomla website which can be found in the configuration.php file (it is the value of the variable $dbprefix).

  • Finally, you should optimize the tables (where you deleted the rows from) by issuing the following queries:

    OPTIMIZE TABLE `#__content`;
    OPTIMIZE TABLE `#__categories`;
    OPTIMIZE TABLE `#__assets`;

  • That’s it!

Again, we recommend you thoroughly think out your decision before deleting all the articles on your Joomla website. If your decision is irreversible and you really want to delete your content, then all you need to do is to execute the above queries. If you need help, then go ahead and contact us. Our work is efficient and professional, we are super dedicated to our customers, and we don’t charge much!

Yet Another Reason for a Blank Page on a Joomla Website

Blank pages on Joomla sites: we thought we have seen them all. But today, we discovered that there might still be a few reasons that we don’t know of that cause blank pages on Joomla sites. Here’s what happened…

A new client called us this morning and told us that he was seeing a blank page on the frontend of his Joomla website. When this happens, we usually set the $error_reporting value in the configuration.php (which is located under the root directory of the Joomla website) to Maximum (usually it’s set to Default or None). When we did that, we saw the following error:

Notice: unserialize() [function.unserialize]: Error at offset 254340 of 254505 bytes in /home/[username]/public_html/libraries/joomla/cache/controller/page.php on line 89

Hmmm… This is a notice, yet it seems to have the effect of a fatal error (we wonder who had the idea of calling such an error as fatal – we’re pretty sure that it’s not fatal, because we’ve seen this error many, many times and it hasn’t killed anyone so far!) because of the blank page. In any case, the word cache in the notice gave us an idea of clearing the cache of the Joomla website. And so we did that by logging in to the backend, and then clicking on System -> Clear Cache, and then selecting all the cached items by clicking on the checkbox next to Cache Group, and then finally clicking on the Delete button on the upper left corner.

Guess what? Clearing the cache fixed the problem!

But what was the problem?

Well, the problem was that the server had some stability issues – and those stability issues caused a cache file not be serialized properly, which meant that Joomla was unable to unserialize the cached file, which meant that Joomla was unable to display the cached information, which resulted in a blank page.

But isn’t that a major stability issue?

In our opinion, Joomla must re-cache the file if the cached file is corrupt (Joomla is able to know that because PHP throws a notice when the cached file is corrupt). Nevertheless, this doesn’t mean that this is a major stability issue. In fact, we have been working on Joomla for nearly a decade and this is (perhaps) the first time that we see this error, since this error happens when Joomla is trying to write the cached file to the filesystem but the operation gets interrupted because of a major stability issue on the server (very rare).

If you see this problem on your Joomla website, then try clearing your Joomla cache and see if it fixes the problem. If it doesn’t, then it might be that there is more to the issue and we recommend that you contact us to resolve the problem. Keep in mind that our super affordable fees apply!

Using Sphinx to Substantially Enhance the Speed of the Joomla Search

Warning: This is an extremely advanced post that requires some serious system administration skills and some serious programming skills. If you feel you’re not up to it, then don’t do it! Ask some Joomla experts (such as, ahem, us!) to do the implementation for you – you’ll save time and your blood pressure will not suffer!

Warning: The implementation in this post requires you to modify a core Joomla file, which means that you will have to ensure that new updates won’t overwrite your modification. If you’re not comfortable with core changes then please contact us and we can develop a new search plugin for you instead of modifying the core plugin.

Note: If you want to rebuild all the Sphinx indexes from scratch, then you will need to stop the searchd listener first using the following command: /usr/local/sphinx/bin/searchd --stop .

Note: Our solution will work. We suggest you don’t deviate from the guide below until everything works for you, and then you can try experimenting with different settings.

Note: While the process below is straightforward and safe, we suggest you take a backup of your whole server before doing anything. Well, at least you should take a backup of your filesystem, your database, and your my.cnf file (this is the MySQL configuration file which is located under the etc folder).

Final note: Every step below is essential for Sphinx to work! You can’t eliminate any steps from our guide and expect that Sphinx works for you. You have been warned!

Let’s face it: Joomla’s search is inefficient – and by inefficient we mean that searches start taking ages once you have over 20k articles on your website. Since we have many clients who are using Joomla and who have a lot of articles on their sites, we have decided to do something about the search! We have decided to use Sphinx…

We first learned about Sphinx a couple of years ago when a large company contacted us and told us that they would like to integrate Sphinx on a mega Joomla website in order to speed up the searches. At that time, our plates were full and we didn’t have time to take on such an endeavor, especially since we have never done it before, so we politely refused (that’s unlike us, but sometimes there is no other option). We were curious nonetheless, and we kept the name Sphinx in the back of our minds!

Fast forward to a couple of weeks ago, when we felt obliged that we had to do something about the search performance (or un-performance, that is) for our major clients’ sites! The searches were just taking too long and causing load issues on the server. It was our duty to do something! And so we thought about giving Sphinx a shot, and boy we were impressed and we never regretted it! Yes – it was a frustrating and downright scary experience, but the end results were astonishing! And since we love sharing and we really want Joomla to become the best CMS ever, we have decided to write a post that will fully explain how we did it!

Are you ready for this super-adventure? Let’s start!

How to install Sphinx on CentOS

Before doing anything, we needed to install the Sphinx search engine on CentOS (CentOS is the underlying OS for mostly all WHM/cPanel powered servers and it is the OS that most of our clients use). Here’s what we did:

  • We logged in to the shell (ssh) as root.
  • We issued the following commands (in order):

    cd /usr/local
    mkdir sphinx
    cd sphinx
    wget http://sphinxsearch.com/files/sphinx-2.2.6-release.tar.gz
    tar -zxf sphinx-2.2.6-release.tar.gz
    cd sphinx-2.2.6-release
    mv * /usr/local/sphinx/
    cd ..
    rm -Rf sphinx-2.2.6-release
    rm -Rf sphinx-2.2.6-release.tar.gz
    ./configure --prefix=/usr/local/sphinx --with-mysql
    make
    make install

  • The above commands installed Sphinx on the server. Note that some of them take a long time to execute, so be patient!

Despite Sphinx being installed on the server (yuppy!), we still couldn’t do anything with it because we needed to configure the Sphinx Index

How to configure the Sphinx index for Joomla’s search

Sphinx is a powerful database engine – but it can’t do much without an index. In this context, an index is a collection of data stored in a specific way for fast retrieval. In the case of a Joomla search, an index consists of a MySQL query that will grab the id, the title, and the introtext and the fulltext (concatenated as text) from the #__content table.

In order to create the index, we needed to first create the Sphinx configuration file. Here’s how we did it:

  • While still logged in to ssh, we issued the following command:

    vi /usr/local/sphinx/etc/sphinx.conf

    The above command will open a new file ready to write using the vi editor (you can use nano, but we’re used to vi – it’s more nerdish).

  • We then clicked on i (yes, the letter i on the keyboard), and then pasted (you can paste in vi, and in linux in general, by just right-clicking) the following code (for your convenience, we have added extensive comments next to each line in order to explain what each line does):

    # source: The source will the a SQL query on the #__content table. The SQL query will be consist of the id, the title, the introtext, and the fulltext.
    # We must specify the connection parameters to the MySQL database, which can be found in the configuration.php file which is located under the root directory of your Joomla website.
    # Note that we are assuming that your MySQL host is localhost (in other words, your MySQL database server resides on the same server as your Joomla website). If this is not the case, then you will need to change localhost and make it point to the right host of your MySQL database server.
    
    source joomla_search_source
    {
    	type		= mysql # Do not change this
    	sql_host	= localhost # You will need to change this if your MySQL server resides on a different server.
    	sql_user	= [your-joomla-db-user] # The username which has at least read access to your MySQL database. No brackets.
    	sql_pass	= [your-joomla-db-password] # The password of the aforementioned username. No brackets.
    	sql_db		= [name-of-your-joomla-database] # The name of your Joomla database. No brackets.
    
    	# Make sure you only include the fields that are being used in your searches - otherwise, you will inflate the size of the index and slow down the searches.
    	# We will just index the id, the title, and a concatenation of the introtext and the fulltext.
    	# Note: You should replace #__ with your table alias, or else the below query (and the index) will not work!
    	sql_query	= SELECT `id`, `title`, CONCAT(`introtext`, `fulltext`) AS `text` FROM #__content WHERE `state`=1
    }
    
    # index: Here we define the parameters of the index created by Sphinx. We will call this joomla_search.
    # If you want to create more indexes then you will need to create another source (like the one above) and another index (like this one) and ensure that the source attribute in the index settings points to the name of the source defined in your source.
    # Note: We initially called the index joomla-search, but everytime we tried to test a query we got the following error: Query failed: unknown local index 'joomla' in search request. It turned out that you cannot have a hyphen (a dash) in the name of the Sphinx index, so we changed it to an underline.
    
    index joomla_search
    {
    	source		= joomla_search_source # Note that the source is the same one specified as the name of the source above.
    	path		= /usr/local/sphinx/var/data/joomlasearch # The indexed data will be stored in this location.
    	min_word_len	= 4 # Sphinx will ignore any words (in the search string) that are less than 4 characters long. You might want to change this to 3 if you have keywords which are 3 characters long. If not, then it's better to leave it as it is as smaller keywords tend to be costly on any search engine.
    }
    
    # indexer: These are the settings of the indexer. Here we just want to increase the memory limit allocated to the indexer, which defaults to a very low value (32 megabytes).
    indexer
    {
    	mem_limit	= 1024M # 1024 megabytes is a comfortable number. If you set mem_limit to a low value then the indexing can take a long a time, if you set it too high then MySQL can timeout.
    }
    
    # searchd: The search engine paramters.
    searchd
    {
    	log		= /usr/local/sphinx/var/log/searchd.log # This is the location of the Sphinx log - logs are always good when something goes wrong.
    	max_children	= 50 # Leave this as 50.
    	pid_file	= /usr/local/sphinx/var/log/searchd.pid # No need to change this at all.
    }

  • We saved the configuration file by typing the following (while still in vi):

    :wq!

Now that we have created the configuration file for the index, we had to build the index…

Building the Sphinx index

The configuration file was ready, so now it was time to build the index. We built it using the following shell command:

/usr/local/sphinx/bin/indexer --config /usr/local/sphinx/etc/sphinx.conf --all

The above command generated the following output:

Sphinx 2.2.6-id64-release (r4843)
Copyright (c) 2001-2014, Andrew Aksyonoff
Copyright (c) 2008-2014, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file ‘/usr/local/sphinx/etc/sphinx.conf’…
indexing index ‘joomla_search’…
WARNING: Attribute count is 0: switching to none docinfo
collected 54142 docs, 154.4 MB
sorted 15.1 Mhits, 100.0% done
total 54142 docs, 154423963 bytes
total 19.459 sec, 7935558 bytes/sec, 2782.25 docs/sec

When we saw the output above, we were rejoiced! It seemed that our configuration file worked and Sphinx was able to index our query! We’re close, very close!

Starting the searchd listener

Now that Sphinx indexed our query, we were ready to start the searchd listener. searchd is a process that will listen on port 9312 for any Sphinx queries and will reply back with results. In order to start searchd, all we needed was to issue the following shell command:

/usr/local/sphinx/bin/searchd

Here’s what we saw when we ran the above command:

Sphinx 2.2.6-id64-release (r4843)
Copyright (c) 2001-2014, Andrew Aksyonoff
Copyright (c) 2008-2014, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file ‘/usr/local/sphinx/etc/sphinx.conf’…
listening on all interfaces, port=9312
listening on all interfaces, port=9306
precaching index ‘joomla_search’
precached 1 index in 0.020 sec

Now we have Sphinx running and listening! Hurray! But, before moving to anything else, we needed to make sure that 1) new data will be indexed every midnight and 2) the listener will run when the server is rebooted.

In order to accomplish that, we did the following:

  • We ran the following command from the shell:

    crontab -e

  • We added the following lines to the end of the cron file:

    0 0 * * * /usr/local/sphinx/bin/indexer --config /usr/local/sphinx/etc/sphinx.conf --rotate joomla_search > /dev/null 2>&1
    @reboot /usr/local/sphinx/bin/searchd

    The first line will rotate the index at midnight (in other words, it will re-index the SQL query), and the second line will ensure that the searchd process runs automatically when the server reboots. Both lines are very important because the first is responsible for keeping the index up-to-date while the second is responsible for ensuring that searches still work when the server is restarted.

  • We saved the cron using CTRL + o, and then exited using CTRL + x.

Now, that we have everything setup, all that we needed to do was to modify the Joomla search plugin to use Sphinx! Here’s how we did it:

  • We opened the file content.php located under the plugins/search/content folder.
  • In the function onContentSearch, we replaced everything above the line $results = array(); with the following code:

    $db = JFactory::getDbo();
    $app = JFactory::getApplication();
    
    require_once JPATH_SITE . '/components/com_content/helpers/route.php';
    require_once JPATH_ADMINISTRATOR . '/components/com_search/helpers/search.php';
    
    require('/usr/local/sphinx/api/sphinxapi.php');
    $sp = new SphinxClient();
    $sp->SetServer('localhost', 9312);
    $sp->SetMatchMode(SPH_MATCH_ALL);
    $sp->SetArrayResult(true);
    $sp->SetLimits(0,1000);
    $limit = $this->params->def('search_limit', 50);
    
    
    switch ($ordering)
    {
    	case 'relevant':
    		break;
    	default:
    		$sp->SetSortMode( SPH_SORT_EXTENDED, 'id DESC');
    		break;
    }
    $results = $sp->Query($text, 'joomla_search');
    $results = $results['matches'];
    $list = array();
    for ($i = 0; $i < count($results); $i++){
    	$sql = "SELECT a.title, a.created, CONCAT(`introtext`, `fulltext`) AS `text`, c.title AS `section`,  CONCAT(a.`id`, ':', a.`alias`) AS `slug`,  CONCAT(c.`id`, ':', c.`alias`) AS `catslug` FROM #__content a, #__categories c WHERE a.catid = c.id AND a.id='".$results[$i]['id']."'";
    	$db->setQuery($sql);
    	$list[] = $db->loadObject();
    }
    $rows[] = $list;

  • We uploaded the file back and we tested the search functionality on the website.

  • We were amazed by the speed difference: our search results were returned instantly on a website with more than 50k articles! Wow - just wow! Oh, and what was even better is that we stopped seeing queries in the slow query log and, even better yet, the load dropped significantly!

So, how long did it take us to implement Sphinx for Joomla's search for the first time?

Well, about 6-7 days (because of the many trials and errors)! But we've learned a lot (again, we have never used Sphinx before) and we have documented the whole process so that you, our precious reader, can do it (hopefully) in less time and less hassle!

We hope that you found our guide useful and you were able to successfully integrate Sphinx with Joomla's search by following it. If you're having problems with the installation of Sphinx or with the integration with Joomla's search, or if the whole thing is just a bit over your head, then please contact us. We will do the work for you in no time and for a very affordable fee!

“Could not start mail function” Error when Registering on a Joomla Website

A new client called us this morning and told us that whenever someone tries registering on his website, he gets the following error:

Could not start mail function

The client was frustrated because he contacted two firms before contacting us, and both of these firms weren’t successful in resolving the problem.

So, the first thing that we tried to do was to reproduce the problem, which was easily done… We did notice though that the registration form wasn’t a basic Joomla registration form, but it was an RSForm which was connected to the Joomla’s users table through the RSForm Joomla User Registration Plugin (in other words, when someone fills in the RSForm registration form and submits it successfully, a profile for that person will be created in the users table).

Initially, we thought the problem was related to the server’s environment, and so we started digging in that direction. But we then ruled out this possibility because we were able to send emails to any email account from a PHP script that we created. We then looked at the configuration settings and we noticed that the Mailer was set to PHP Mail, which was what it should be… We then started testing the form more thoroughly, and we noticed that for some reason, some emails did get through and some emails didn’t – but in any case, the form submission took about 15-20 seconds. Hmmm…

We kept working on the issue for a couple of hours until we discovered that the problem was that the registration form was trying to send emails on behalf of emailaddress@aol.com (we thought it was sending emails on behalf of a local email, but we were wrong). As we have explained before, mails can be blocked by the receiving server if the sender IP is not in the SPF records. Clearly, aol.com did not have the IP of our client’s server in their SPF records, and that’s why the mails were failing.

But, why was it that some mails were going through?

Well, it was because some mail servers do not do any SPF check on the sending server; they just allow any email to go through.

Isn’t the error “could not start mail function” a bit misleading?

Yes – it’s very misleading, but the thing is that PHP usually doesn’t wait for a response from the receiving server, but, in some cases (depending on the sending server environment), it does (which is the case for our client, hence the 15-20 seconds it takes to submit the registration form), and so the PHP mail function fails, and so the wrapping Joomla function returns this erroneous error (isn’t that an oxymoron?).

So, what did we do to fix the problem?

  • We logged in to the backend of the Joomla website.
  • We navigated to Components-> RS Form Pro -> [Form Name] -> Properties -> Admin Emails

  • We then changed the From email to admin@ourclientjoomlawebsite.com

  • We saved the form and that fixed the problem!

What if I’m not using RSForm?

Well, if you’re not using RSForm and you’re just using Joomla’s built-in registration form, then make sure that the From Email under the Mail Settings in the Global Configuration settings is set to a local email if you’re using PHP Mail as the Mailer (by local email we mean an email which domain has the IP of your server in its SPF records).

If you have the same problem when people are trying to register on your website then make sure that the IP of your mail server is allowed to send emails on behalf of the domain in the from email. If you are sure that everything’s OK on your end but you are still getting this error, then you can always contact us and we’ll solve the problem for you in no time and for a very, very affordable fee!

“UID of script file.php is smaller than min_uid” Error on a Joomla Website

An international news agency using Joomla to power one of their websites called us yesterday. They told us that they’re seeing a blank page, and the problem turned out to be related to sh404SEF’s cache (the cache file was exhausting all the memory that the system has and throwing this error).

Due to the nature of their environment and their security policies, we weren’t able to work directly on their live system; we worked with a system administrator by phone and that system administrator executed the work on the live site.

In any case, we tried to get some information about their server (mainly related to the PHP’s memory limit), and so we asked the system administrator to create a file called file.php that gets the necessary information (it was mainly getting the memory limit using the ini_get(‘memory_limit’) function). We asked the system administrator to place that file under the root directory of the site.

But, when we accessed the file from our browser (using the following URL: http://www.ourclientjoomlawebsite.com/file.php), we got the following fatal error:

“UID of script “/www/ourclientjoomlawebsite/file.php” is smaller than min_uid”

Huh?

We did a quick research on the error above and it turned out to be related to the file’s ownership. The file was owned by root (because it was created from the shell by the system administrator who was logged in as root), and yet the environment was using suPHP (which cannot directly execute files owned by root). Changing the file’s ownership from root to the Apache user fixed the problem!

If you’re having the same problem with one of the files on your Joomla website, then make sure that file is owned by the same user who owns the rest of the files on your Joomla website. If changing the ownership did not fix the problem for you, then please contact us. We will fix the problem for you for a very affordable fee and in as little time as possible!

What to Do When Your Joomla Website Is Blocked by HP’s Tipping Point

One of our very large clients sent us an email stating that the doctors working at an extremely high profile hospital were unable to access their website (our client’s Joomla website). The email stated that the website was timing out for them.

At first, we thought it was a browser issue (since large companies tend to use the same browser, and it’s often an old version of Internet Explorer with weird patches), but then we ruled it out because the IT director (who communicated directly with us), was also unable to to access the website from a different browser. The staff were able to access the website from their mobile devices (which were using a different network than the hospital’s network), but they weren’t able to access the site from the hospital’s network.

We then asked the IT Director to check with his networking team whether the website was blocked at the firewall level, and the immediate feedback was that the website wasn’t blocked. Perplexed, we started searching for another cause of the problem, and so we checked our client’s logs and we noticed that the doctors were hitting the website, but for some reason, they were getting a 301 redirect and that’s it – the connection ends and the browser stops asking the server for further content. We scavenged through all the security modules that are installed on the server thinking that that network was blocked somewhere, but we were unsuccessful at finding anything.

We then asked for a traceroute, and we were sent one immediately, but, to our disappointment, the IT Director was able to successfully tracert to our client’s website. Hmmm… The IT Director then asked us for a traceroute from our server to theirs, and something interesting happened: the traceroute failed as soon as it reached their network, which means that they’re not accepting traffic from our (client’s) server!

A few minutes later, the IT Director emailed us and told us that they resolved the problem on their end, stating the following reason: “HP has an IPS (Intrusion Prevention System) application named Tipping Point. A reputation database (refreshed from HP’s dvlabs feed) is integrated into that IPS. Your website had a bad rating and that’s why it was blocked. We resolved the problem by locally whitelisting your website.”

He then sent us the following snapshot:

dvlabs Change of Score

Figure 1: dvlabs Change of Score

As you can see from the above picture, the website has been labeled as carrying a “Network Worm” for nearly a month. Clearly, we needed to contact HP’s tipping point about this issue (we were confident that the website was clean). Here’s what we did:

  • We called HP’s Tipping Point Support at +1 866 681 8324 (you can find the global phone numbers here).
  • We supplied the agent with the needed information (our client’s website, the account information of our client’s client) and we asserted that the website was very clean and not hacked.

  • The agent was very courteous and told us that they get a lot of false positives on the dvlabs database, and that the reason why our client’s website was categorized as a “Network Worm” is because they possibly received an abnormal amount of traffic from the website at one point or another. The agent confirmed that he will escalate that the problem to the dvlabs team, and, if the website is proven to be clean, then it will be removed from their blacklist and the change will be reflected once their data feed is refreshed.

  • We thanked the agent who told us to wait for the email. That was back on Friday (January the 2nd).

  • On Tuesday, we received an email from the agent apologizing for the wait, and telling us that the website has been removed from the blacklist.

  • We rejoiced and we informed our client and the IT director of the hospital!

As you can see, it is not that hard to remove a website from HP’s Tipping Point blaklist (e.g. the dvlabs database) – it just needs a bit of patience. It is also not a bad experience (we were very surprised with HP’s pro-activeness, professionalism, and friendliness towards this whole issue).

If some of your clients cannot see your website because it has been blacklisted by HP, then make sure it is clean, and then contact them (they do answer the phone immediately). If you need help in the process, then please contact us. We have done it before (and it did work), we are resilient, we work hard, and we don’t charge much!

A Menu Item with “Registered” Access Doesn’t Mean that the Article Is Private

One of Joomla’s biggest misconceptions is that if someone wants to make an article private (e.g. only allow access to that article for registered users), then all that needs to be done is to point a menu item to that article and change the Access value of that menu item to Registered and that’s it! Now the article is private and only registered users will be able to access it – of course, the proof is that the link to that article, which is http://www.[joomla-website].com/my-article.php will only work for registered users.

But… Assuming the article’s ID is 123, then that same article can be accessed using the following link: http://www.[joomla-website].com/index.php?option=com_content&view=article&id=123, and this link can be accessed by anyone, and by anyone we literally mean anyone – even a person who’s not even registered to the website!

How is that? Well, because this is a direct link that doesn’t have the Itemid of the menu item that only allows access for registered users. In fact, it doesn’t have any Itemid (an Itemid is the equivalent of a menu item ID).

So, what is the solution if one wants to make an article private?

Well, the only solution is to get all the Itemids (yes, it has an s because it’s plural since an article can have more than one menu item pointing to it), and then check if any Itemid is set to have a Registered access, and if that’s the case, then the code should check whether the person is logged in or not, if not, then the person should be redirected to the 404 page. If the person is logged in, then he should see the page.

So, where should this solution be implemented?

The solution can be implemented in either a System plugin or through a core modification. Naturally, we recommend the former option since it’s better, more reliable, and future-proof (e.g. update-proof).

If you’re trying to make your articles really private (e.g. only allow access to your articles for registered users), and if you need help in making it possible, then please contact us. We can do the work for you in no time and for a very affordable fee!

Joomla’s Cache Is Deleted on Article Update: How to Fix

Note: This is an extremely advanced post that requires some serious knowledge with Joomla as well as more than average programming skills. If you feel that it’s a bit over your head then you should ask some Joomla experts to do it for you (like, ahem, us) since caching is a very delicate feature in Joomla.

A client of ours were (or should it be “was” instead of “were” despite the fact that we’re talking about a company? English grammar can be very tricky!) encountering very high load issues on their server only during working hours. They were using Joomla’s core (and not K2) for content management. After a very long investigation of the issue (at first we thought the whole issue was related to an unoptimized assets table, but we quickly ruled this out out as the assets table is not updated on article update), we narrowed down the problem to one word: caching – or lack of caching thereof.

Let us explain… They were using Conservative Caching on their website, and during working hours, their editors were frequently creating/updating articles on the website, and every time they created/updated an article the whole cache was cleared!

But why was the cache cleared?

That is the million dollar question, and the million dollar answer is that the method cleanCache in the ContentModelArticle class (which is defined in the file article.php file located under the administrator/components/com_content/models folder) is called when an article is inserted or updated. Here’s the function, in case you’re wondering how it looks like:

protected function cleanCache($group = null, $client_id = 0)
{
	parent::cleanCache('com_content');
	parent::cleanCache('mod_articles_archive');
	parent::cleanCache('mod_articles_categories');
	parent::cleanCache('mod_articles_category');
	parent::cleanCache('mod_articles_latest');
	parent::cleanCache('mod_articles_news');
	parent::cleanCache('mod_articles_popular');
}

As you can see, the function deletes all the content cache (the cache generated by the main content component and all the content modules). This function is invoked on every update and every insert. Obviously, on a large site relying on caching to sustain performance, that function can cause some serious load issues on the server that can potentially crash the site.

But why does the function clear all the content cache?

We have no idea. We think that this code is there because it was the easiest way to make sure that the site displays the most up-to-date content with as little code (and work [wink wink]) as possible. Unfortunately, the easiest way is far from being the optimal way. The optimal way consists of only deleting that cache pertinent to the updated article (and not the whole cache). We really wonder why nobody has ever complained about this before, since what this function does is not acceptable for any large Joomla website.

So how did we solve the problem?

Solving this problem was hard – like really really hard! We modified the function cleanCache to only delete the cache of the homepage, the updated article, and all the category blog pages for the category that this particular article belongs to. In the process of modifying the function, we created an auxiliary method called myMakeID that returns the ID of the cache to be deleted (the myMakeID method is a variation of the makeId static method on the JCache class which is defined in the cache.php file which is located under the libraries/joomla/cache folder). Here is the updated cleanCache method, as well as the auxiliary myMakeID method:

protected function cleanCache($group = null, $client_id = 0)
{
	//let's first get the Item ID of the category (which should be always in the POST)
	$catId = (int)$_POST['jform']['catid'];
	
	if (isset($_GET['id']))
		$id = (int)$_GET['id'];
	else
		$id = 0;
	
	$db = JFactory::getDbo();
	
	//now let's get the Item ID for that category
	$sql = "SELECT id FROM #__menu WHERE `link` ='index.php?option=com_content&view=category&layout=blog&id=".$catId."'";
	$db->setQuery($sql);
	$itemId = $db->loadResult();
	
	//now let's get the Item ID for the homepage
	$sql = "SELECT id FROM #__menu WHERE `home` =1";
	$db->setQuery($sql);
	$homeItemId = $db->loadResult();
	
	
	$conf         = JFactory::getConfig();
        $platformDirs = FOFPlatform::getInstance()->getPlatformBaseDirs();

	$options = array(
		'defaultgroup' => ($group) ? $group : (isset($this->option) ? $this->option : JFactory::getApplication()->input->get('option')),
		'cachebase'    => ($client_id) ? $platformDirs['admin'] . '/cache' : $conf->get('cache_path', $platformDirs['public'] . '/cache'));

	$cache = JCache::getInstance('callback', $options);
	
	//first delete the cache for this particular article
	if (!empty($id) && !empty($catId)){
		$makeID = $this->myMakeID($id, $catId, $itemId);
		$cache->remove($makeID, 'com_content');
		$cache->remove($makeID, 'mobile-com_content');
	}
	if (!empty($catId)){
		$makeID = $this->myMakeID($catId, NULL, $itemId, 'ContentViewCategory', 'category', NULL, 'blog');
		$cache->remove($makeID, 'com_content');
		$cache->remove($makeID, 'mobile-com_content');
	}
	if (!empty($homeItemId)){
		$makeID = $this->myMakeID(NULL, NULL, $homeItemId, 'ContentViewFeatured', 'featured', NULL, NULL);
		$cache->remove($makeID, 'com_content');
		$cache->remove($makeID, 'mobile-com_content');
	}
	
}

protected function myMakeID($id, $catId, $itemId, $classView = 'ContentViewArticle', $view='article', $format = 'html', $layout = NULL)
{
	$safeuriaddon = new stdClass;
	if (!is_null($catId))
		$safeuriaddon->catid= (int)$catId;
	else
		$safeuriaddon->catid= NULL;
	if (!is_null($id))
		$safeuriaddon->id= (int)$id;
	else
		$safeuriaddon->id= NULL;
		
	$safeuriaddon->cid= NULL;
	$safeuriaddon->year= NULL;
	$safeuriaddon->month= NULL;
	$safeuriaddon->limit= NULL;
	$safeuriaddon->limitstart= NULL;
	$safeuriaddon->showall= NULL;
	$safeuriaddon->return= NULL;
	$safeuriaddon->filter= NULL;
	$safeuriaddon->filter_order= NULL;
	$safeuriaddon->filter_order_Dir= NULL;
	$safeuriaddon->{'filter-search'} = NULL;
	$safeuriaddon->print= NULL;
	$safeuriaddon->lang= NULL;
	$safeuriaddon->Itemid= (int)$itemId;
	$safeuriaddon->format= $format;
	$safeuriaddon->option= 'com_content';
	$safeuriaddon->view= $view;
	$safeuriaddon->layout= $layout;
	$safeuriaddon->tpl= NULL;
	return md5(serialize(array(md5(serialize($safeuriaddon)), $classView, 'display')));
}

Adding the above code to the aforementioned article.php model file will solve the problem, but keep in mind that this is a core modification (which means future updates to the site may wipe out your changes). If you’re not at ease doing this modification yourself then please contact us. We can do it for you in as little time as possible, for a very affordable fee, and with never-seen-before (even on TV during the 1990’s) enthusiasm mixed with a healthy dose of humor!

How to Change the Browser Title on the Homepage of a Joomla Website

Note: This post is aimed at Joomla beginners – make that absolute Joomla beginners! We’ve been writing posts for extremely advanced Joomla developers for a long time now and we thought writing a post for absolute beginners would be the right thing to do on New Year’s Eve (Happy New Year, by the way)!

We get a couple of calls every week from (new) Joomla administrators who want to change the browser title on the homepage of their Joomla website, but they don’t know how, and so we do it for them. Here’s how:

  • We login to the backend of their Joomla website.
  • We click on Menus on the top navigation, and then we click on the menu that has the homepage (the menu with the home picture next to it).

  • Once we’re in the menu that has the homepage (there’s only one home menu), we search for the home menu item. We do this by clicking on the Home header twice (to show the home menu item first). The home menu item is the menu item that has an orange star under the Home header.

  • We then click on the home menu item, and then we click on the Page Display tab.

  • We then change the value of the Browser Page Title to the new title.

  • We click on Save & Close on the top left.

  • We clear the cache by going to System -> Clear Cache, and then clicking on the top checkbox, and finally clicking on the Delete button on the top left.

  • That’s it!

In most cases, the above will solve your problem. But if it doesn’t, then it’s possible that the browser title is controlled by the template or by a 3rd party extension. If this is the case, then please contact us. We will be able to do it for you in very little time and for a very affordable fee!

How to Redirect Your Old Links to Your New Links After Enabling Apache’s URL Rewriting in Joomla

A new client of ours has been using Joomla for a long time (since Joomla 1.5 was introduced), but has never, ever used Apache’s URL Rewriting. She has been using Joomla’s SEF all the time, but she never enabled the Use URL Rewriting feature. She knew that enabling URL rewriting can boost the search engine rankings for her website, but she was afraid to do it for fear of “duplicate content”, and she is right. Here’s why:

A Joomla SEF link, but not a URL rewritten link, looks like the following:

http://www.yourjoomlawebsite.com/index.php/your-link[.html]

The same link with URL rewriting enabled will look like the following:

http://www.yourjoomlawebsite.com/your-link[.html]

As you can see, index.php/ is missing from the second link. But that’s OK, since both links work, right?

Actually wrong, this is exactly why our client (and rightly so) was hesitant to switch to URL Rewriting. She didn’t want to have two working links pointing to the same page, because if she did, her website would suffer from content duplication issue (when it comes to Google’s indexing), not to mention that the strength of the links on her website would be diluted (because, again, she would have 2 links pointing to the same page).

So, how did we solve this problem?

Fortunately, at itoctopus, we have a solution for everything Joomla, and we were able to address her problem by adding the following lines to the .htaccess file (which is located under the root directory of her Joomla website) just below the RewriteEngine On line:

RedirectMatch 301 /index.php/(.*).html /$1.html
RedirectMatch 301 /index.php/(.*) /$1.html

The above lines ensure that any link that has index.php in it will automatically redirect to the new link (that doesn’t have index.php in it). The 301 number in each line ensures that we are passing the link strength (e.g. the link juice) to the new page. Note that the .html at the end of each line should be removed if Adds Suffix to URL is disabled in Joomla’s configuration settings. (In the latter case, the second line will have no effect and must be removed because it’ll be exactly the same as the first line.)

As you can see from the above, it is extremely easy to enable Apache’s URL Rewriting on a Joomla website without losing any rankings provided you add the above 2 lines. But, if you’re one of those Joomla administrators who prefer not to touch that dreadful .htaccess file, then please contact us and we’ll do it for you! Our fees are affordable, our work is professional, and we are the friendliest programmers in the whole wide world!

Joomla Is Making the Same Mistake, Again!

Joomla 1.5, probably the most used Joomla version ever, officially died back in 2012. Back then, many Joomla administrators were faced with a dilemma: Should we migrate our Joomla website(s) to 2.5? Or should we switch to another CMS? The decision was not easy, because in many cases, the costs of migrating to Joomla 2.5 were more or less the same as those of switching to a completely different CMS. Unfortunately, those administrators didn’t have the option of simply continuing to use Joomla 1.5.26, because a nasty security exploit was discovered nearly 1 week after Joomla 1.5.26’s life ended, and so they had to choose between the lesser of two evils, and for many of them, the lesser of two evils was switching to another CMS. Needless to say, Joomla’s competitors gained many new converts at Joomla’s expense.

In our opinion, had Joomla created a “one-click” migration script to migrate sites from Joomla 1.5 to Joomla 2.5, the whole world would’ve been a better place (well, not the whole world, but most of it!). Instead, Joomla took an extremely passive stance and it didn’t even release a single official tool for any type of migration from 1.5 to 2.5 – Joomla didn’t even create a tool to migrate core content (such as articles, categories, and menu items), the only thing that Joomla did to address this issue was to point its users to 3rd party tools that required some considerable voodoo skills for them to work as they should. The migration process from 1.5 to 2.5 left a very sour taste in the mouth of everyone who used Joomla…

Fast forward to now, and Joomla is repeating the exact same mistake of 2012. Joomla 2.5’s support will end by the end of this month (December 2014) and there are no real automated tools to migrate from Joomla 2.5 to Joomla 3.3. Yes, Joomla insists that it’s a mini-migration from 2.5 to 3.3 and that it is extremely easy and that it takes a few minutes, but, the truth, it is not. In fact, it is far from being a mini-migration (it is a real migration), and it takes more or less the same time to migrate from Joomla 1.5 to Joomla 2.5. Let us give you an example…

At itoctopus, we only use our tools to migrate from one version of Joomla to another (note that we modify the code of these tools for each website that we migrate), but, we got a very small website (less than 100 pages and a couple of well-supported 3rd party extensions) to migrate this past week, and so we thought we can just do the migration by following Joomla’s guide, and so we did. Here’s what we saw when we finished the 5 minute migration…

Table ‘#__postinstall_messages’ doesn’t exist SQL=SHOW FULL COLUMNS FROM `#__postinstall_messages`

Hmmm…. The table #__postinstall_messages wasn’t created. Odd. So we created it manually in phpMyAdmin. We thought that would fix the problem, but when we refreshed the homepage, here’s what we saw:

Notice: Undefined property: ContentController::$input in /home/[username]/public_html/administrator/components/com_content/controller.php on line 39

Fatal error: Call to a member function get() on a non-object in /home/[username]/public_html/administrator/components/com_content/controller.php on line 39

Not a problem. We deleted the controller.php file, but then we got another error…

Fatal error: Call to undefined method JFactory::getXMLParser() in /plugins/system/nonumberelements/helpers/parameters.php on line 93

OK. That is a 3rd party extension – let’s just disable it and see if the website works…

A quick refresh threw another error in our faces…

500 – JHtmlBehavior::mootools not found.

While becoming a bit frustrated, we fixed the above problem by including MooTools the Joomla 3.3 way in the template.

Refreshing the homepage generated another error that had to do with a specific extension, so we thought, we can just update that extension from Joomla’s backend (we were able to login to the backend).

Unfortunately, when we clicked on the Updates are available link (located on the left side-panel under the Maintenance section on the homepage of Joomla’s backend) we got the following error:

Unknown column ‘a.extra_query’ in ‘field list’ SQL=SELECT DISTINCT a.update_site_id, a.type, a.location, a.last_check_timestamp, a.extra_query FROM #__update_sites AS a WHERE a.enabled = 1

And so we tried to fix the database by clicking on the Fix button on the upper left corner on the Extensions -> Extension Manager -> Database page, but we got the following error:

Unknown column ‘title_alias’ in ‘#__content’ SQL=ALTER TABLE `#__content` CHANGE `title_alias` `title_alias` VARCHAR( 255 ) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT ”;

At this point, we started browsing the Internet for the best practices for committing suicide painlessly and professionally. But then we thought, while we really wanted to go through this migration the Joomla way, our lives, sanity, and families are much more important, and so we reverted the website to a previous backup, we re-did the migration our way, and this time we were actually able to finish it successfully.

And then we started thinking, we are established Joomla experts and we failed at making the built-in migration work for us when migrating a very, very simple Joomla website. So what about those who are trying to migrate from 2.5 to 3.3 and who are not experts? How will they be able to migrate their sites by themselves? And what if they can’t afford to pay us or other Joomla developers to migrate their sites? Will they stick with Joomla if they can’t? Or will they not?

Obviously, Joomla users are once again at a crossroad, they will either continue with Joomla, or they won’t, and probably only those who are extremely loyal to Joomla will choose the former option, because many Joomla users were burnt back in 2012, and they are less likely to be willing to get burnt a couple of years later, and a couple of years later, and a couple of year later…

So, what can Joomla do?

In order to prevent a massive and continuous hemorrhage of users, Joomla must create a migration tool that actually works, instead of this not even half-baked solution that Joomla has (e.g. by switching from LTS to STS and then updating the website). Failing that, Joomla’s userbase will substantially shrink in 2015-2016, possibly never returning to its 2014 levels anytime in the near or distant future…

Now, if you are trying to migrate your Joomla website but it’s not working out well for you, then please let us know. We will do the migration for you in as little time as possible and for a very affordable price. Oh, and we will ensure that the frontend of the migrated website will look and work exactly like your old website!

Creating New Articles on Your Joomla Website Is Taking a Long Time? Clean Your Assets Table!

We’re not big fans of the #__assets table, but you probably know this already if you’re one of our loyal readers! The #__assets table makes everything on the website slower, and not to mention that minor data integrity issues on that table can lead to large problems on the Joomla website.

But let’s focus in this post on the performance issues caused by the #__assets table, and what to do about them. In short, when you add a new content entry (an article, a category, a module, etc…), Joomla has to update a large chunk of the rows in the #__assets table to insert the new row, which will increase the load on the server. So, the more rows you have in your #__assets table, the more time you’ll have to wait when you save a new article.

If you want proof that the #__assets table is the bottleneck, then here’s an example from the MySQL Slow Query Log from one of the websites that we manage:

# Time: 141212 12:57:49
# User@Host: database-user[database-name] @ localhost [] Id: 282378
# Query_time: 5.492695 Lock_time: 0.000000 Rows_sent: 0 Rows_examined: 2303
use database-name;
SET timestamp=1418252269;
UPDATE #__assets
SET lft = lft + 2
WHERE lft > 648;

The query above took nearly 5.5 seconds and it happened when our client added a new article to their website. That client usually adds about 70 articles/day – which means that they have 70 of these slow queries every day. And it gets worse, since the larger the table is, the more time these queries will take and the more the load there will be on the server (the 5.5 seconds occurred on a table that only had 2,300 rows – imagine what would have happened if that table had 100 times that number of rows – content managers will have to wait 10 minutes after clicking that “Save” button!).

So, what do about this?

Well, the solution is simpler than you can possibly imagine. All you need to do is to regularly delete those rows (in the #__assets table) that are created by adding new articles! This can be done by executing the following query in phpMyAdmin (make sure you backup your database before executing the below query and also make sure you optimize the table after executing the query):

DELETE FROM #__assets WHERE `name` LIKE '%com_content.article.%';

We know, you probably think that we have gone insane and that we’re trying to destroy your website, but this is not the case. These entries have no purpose whatsoever and they are not needed for your website to work. You can really delete them, and once you do, you will notice a substantial drop in your server’s load during working hours and your new articles will save instantly.

Now, you may be wondering, will there be any side effects?

We regularly clean the #__assets table of our major clients, and so far, we haven’t seen a single glitch caused by it. But, if you’re afraid to do the above yourself, then a good idea would be to contact us, we’ll do it for you in no time (we’ll also create a cron job to automatically clean the #__assets table for you at regular intervals) and for a very affordable fee!

“index.php” Should Be the Only Accessible File on Your Joomla Website

Here’s how the majority of malicious users try to hack your beloved Joomla website:

  • They search for an exploit on your Joomla website.
  • They use that exploit to upload a PHP file to your website.

  • They invoke the URL pointing to that file, and by doing so, they create duplicates of the malicious file, hack your website, or send spam.

So, if they upload a file called cms.php, then all they need to do is to go to http://www.yourjoomlawebsite.com/cms.php and it’s over: your website either becomes labelled as a “compromised website” by Google or you will receive the dreaded email from your host telling you that your Joomla website is sending spam and you have 24 hours to address the problem or it will be shut down! At best, your website becomes an incubator of malicious files that will be used by any wannabe malicious user out there.

Fortunately, there’s a simple solution that will address this problem: block all access to all PHP files with the exception of the index.php file: your visitors, and even your administrators, only need access to the index.php file to be able to visit/manage your website. They should never need direct access to any other file on your server.

So, how to only allow access to the index.php file?

This can be done easily by adding the following code to the very beginning of your .htaccess file:

<Files *.php>
	Deny from all
</Files>
<Files index.php>
	Allow from all
</Files>

Once you implement the above, even if the malicious user was able to upload the cms.php file to your Joomla website through a vulnerability, he will not be able to execute it (he will be redirected to http://www.yourjoomlawebsite.com/403.shtml page), which means that he just can’t do anything with the file that was uploaded.

But how will administrators be still able to login?

If you have read our post from the beginning (yes, some people still do that instead of just jumping to the solution), you will see that we said that this trick also works for administrators. But, as you (probably) know, administrators use another index.php, which is the one located under the administrator folder. Luckily, the .htaccess rules are recursive, which means that any index.php file in any directory (and sub-directory) will still work, including the one located under administrator folder. Now, although this is convenient, it creates another problem: what if the attacker uploads a file called index.php, under, let’s say, the media folder? “Obviously, this solution is not bullet proof!”, you might be wondering…

You’re right, it is not bullet proof! It does block the majority of the hacks, but an experienced saboteur can still upload an index.php file instead of a cms.php file and he’ll gain control over your website. To address this problem, you will need to completely deny PHP execution in all the Joomla folders by creating, under each and every folder (with the exception of the administrator folder), an .htaccess file with the following content:

<Files *.php>
	Deny from all
</Files>

This will block the execution of all PHP scripts within those folders, even if they are named index.php. Since we will need to be able to execute the index.php file under the administrator folder to allow for backend access, we can’t add that .htaccess file there, but we will need to add it under all the directories located under the administrator folder.

So, will this solution guarantee the security of my Joomla website?

No – it won’t. Certain exploits will allow malicious users to modify core Joomla PHP files (including the index.php file), so the solution is not a silver bullet. It’s not worthless though, and we think it’s a must, as it is a first level barrier against attacks and will greatly increase the security of your Joomla website.

But what if an extension needs direct access to another PHP file?

There are a few extensions out there that r