xTuple Web Client: It's not just for mobile devices
One of the great misunderstandings about the new xTuple web client is that its primary purpose in life is to provide an interface for mobile devices. This may surprise people, but that was never the primary reason for its creation. In fact, we expect more people to use it on laptop and PC environments than on mobile devices. It has been labeled the “Mobile” client entirely for marketing reasons to differentiate it from the Qt based “Desktop” client. There were actually several goals we were attempting to achieve in development by creating this technology including:
A user interface that could run in any modern browser.
An object oriented paradigm for defining business objects and their corresponding functions.
Improved performance on large data sets.
Unit of Work commit transactions.
System wide support for locking.
More consistent user interface.
More modular design.
Web services support for easier system to system integration.
Native support for column and row level security.
Mobile ready interface that will work on any device including touch devices such as tablets and phones.
That's a lot of goals! Some of them may not make sense to you, so I will describe each of them in more depth.
A user interface that can run in any browser
The benefits of this are probably among the most obvious. The main idea here is system administrators do not have to deploy and update client software on local devices. Users can log in from anywhere using any device with a modern browser.
Cloud ready architecture
I know, I know. The use of the term “cloud” is such an over-hyped and misused buzz word that it causes as much eye rolling as it does excitement. What this really means to us is that we can deploy the software as a service in a way that can scale economically as the user footprint grows, and that works well over the internet where connectivity and bandwidth are often spotty and unpredictable.
An object oriented paradigm for defining business objects and their corresponding functions
I debate whether I should list this item as third, because in my mind it is the most important attribute of the new architecture because object oriented programming (OOP) enables just about everything else on this list. Basically we have introduced a core notion of a model based business object that defines the universal behavior of all objects in the system. This is a radical change from our Desktop client which uses scripting techniques and hand written SQL statements to drive all functionality. What this means is if we want to add a new system wide feature we only have to add the feature in one place and it becomes available everywhere. It also means it is much easier to extend the application with custom functionality, with a much lower probability of breaking the core on the next upgrade. I could write an entire book on this subject, but suffice it to say that many books have already been written on OOP and this is a sea change for xTuple.
Improved performance for large data sets and complex functions
The web client implements a technique known as “lazy loading” when presenting lists of data. What this means is when you bring up a list of what is potentially a million records, only the first few are actually loaded. As you scroll through the list, additional records are loaded a few more at a time in the background. In this way you don't burden either the client or the server with the complete request to query and process a million records.
Like all modern browser based applications, the new client also features ubiquitous use of asynchronous programming. What this means is when a user does something that requires a request to the server to get more information, the client allows the user continue navigating and doing other things until the answer is returned. This is vastly different from a single threaded application like the Desktop client which basically stops all further activity until it receives a response from the server.
Finally, the web client has been optimized to send as little information back and forth as possible. After the application is loaded in your browser, no further scripts or HTML are passed between the client and the server. Only business data is transmitted, and even then we cache and compress data when possible to ensure minimal chatter between the client and the server. These optimizations, coupled with asynchronous programming and lazy loading are reasons why the web client can run just as well from a server in Singapore to the United States as it can on a server in your local region.
Unit of Work Transactions
This is very closely related to object oriented programming, but what it means is when business objects or transactions are committed to the server, they are committed in their entirety. In the case of a complex object like Sales Order this means nothing goes to the server until you click “Save” and when you do the header, line items, comments and everything else gets committed all at once. By contrast in the Desktop client when you are working on a document you are committing bits and pieces to the server. This can lead to problems if you get abruptly disconnected server while working on an incomplete document. Unit of work also enables the next important feature:
One of the things that has dogged us the most over the years is the dearth of inline editing capability in the Desktop client, meaning that people want to edit lists such as Sales Order line items directly in the same form as the header. People don't like to have to open a new form every time they create a line item, enter a comment, or a characteristic. We've created work arounds for this in a few places, but by and large you can't do this across the board without an object oriented foundation that uses unit of work transactions described above. In the web client you will find that all business objects with line items allow direct inline entry on the main workspace for the document, and you can also edit other things like comments and characteristics inline.
System Wide Support for record locking
A classic problem in enterprise applications comes up when you have two people open the same record for editing. What happens when they both make changes and save? Without a record locking system, the second person's changes over-writes those of the first. The Desktop application has record locking on Sales Orders, Purchase Orders and Transfer Orders that disallows the second person from editing until the first person is done. The web client has this behavior on all business objects. This is only feasible with an object oriented architecture.
More consistent user interface
We've had complaints from time to time about some screens behaving different from others. For example on the Desktop client in the Sales Order screen when you click “Save” the screen saves and clears for a new record. In the Invoice screen when you click “Save” it saves and closes. In the web client all screens (called workspaces) have the same controls: “Refresh” requerys the data. “New” will save the current record and clear the screen for the next. “Apply” allows you to save and stay on the same record. “Save” will save the record and send you back to the previous screen in one click.
There other nice consistency improvements, like all lists having the same filtering and sorting mechanism, all layouts following the same style conventions and so forth.
More modular design
When you are dealing with an enterprise application like an ERP system modularity is critical because there is so much variation from business to business. Different businesses, and even different users within a business, need to be able to have the application do different things which usually means adding or removing large swaths of functionality for each implementation. It can also mean doing deep invasive customizations. This all needs to be able to happen in a way that is relatively easy to turn off and on, appears seamless to the end user, and allows the application to be upgradable in the future.
The web client is built as a modular application from the ground up, not as an after thought, and is designed to achieve all theses goals. For example in the web client you can completely turn off CRM at the user level so it does not appear at all. Once again object oriented design enables this capability.
Web services support for easier system to system integration
It's a no brainer these days that any ERP application worth its salt has a web services interface to integrate with other external systems. We have implemented a modern REST interface for just that purpose.
Native support for column and row level security
In the world of ERP security can be a huge issue. It's not unusual to want to disallow users from seeing not just application level functionality but very specific data such as cost data, or rows that don't “belong” to them. The web client was designed with this in mind so whereas in the Desktop client adding and enforcing these kinds of rules can be extremely difficult and painful, doing the same in the web client is trivial.
We haven't done much to implement row level security in the web client outside of what the Desktop client already does (namely “Personal” privileges), but the new eCommerce portal takes extensive advantage of this new capability by doing things like only allowing customers to access “their” Contacts, Orders, and Invoices. This is possible because the privilege rules are enforced at the user level all the way down to the aforementioned REST interface.
Mobile ready interface
Yes, though the purpose of this blog was to downplay this element, making our application mobile-ready was an important goal. Of course tablets and smart phones are ubiquitous now and people expect to be able to access their business system from these devices. Furthermore, they expect to be able to just pick up an iPad or ruggedized tablet and use it to perform inventory functions! This is a big shift from not too long ago when you had to expend huge amounts of money to purchase special software and hardware to implement a wireless inventory management system.
Note, however, one thing that makes xTuple different in this regard leads back to my early claim that we expect as many or more people to use the web client on laptops and desktop computers as we do mobile devices. Most companies write an entirely new native client for each mobile platform. This can lead to disarmingly different user interfaces moving from device to device. Because xTuple's interface is designed to work in all environments, there is no need to retrain people on how to use each device. Also, if you make a customization or add a new extension, that new functionality is available to all platforms and devices, which can be huge.
So there are a lot of great things to talk about on the new web architecture that go far beyond just mobile capability. The amazing thing is, this is just the tip of the iceberg! There's so much more to talk about, but you can only hold attention in a blog for so long. The best way to find out about other improvements is to read about and play with the software.
Read about how its put together here
Play with it here
Sat, 05/10/2014 - 10:20#1
Thanks John for the effort you put into clarifying these things.
It seems to me like I am hearing that going forward the Web interface is the interface of choice for xTuple and the desktop client will eventually disappear. Now eventually may be some time off but as I read this the desktop client development just simply will not allow the sort of interface you are describing and the functionality that goes along with it.
I have no interest in starting any kind of debate about whether this is the "best" approach or not, and by "best" I mean what allows us to sell more systems. After all is said and done selling systems is really the only thing that matters.
I believe it is obvious or at least it should be obvious that there has been great effort by xTuple to expose scripting through the desktop client to allow customization and educate the reseller community on how to us it. Many of us have spent some effort learning scripting to accomplish both simple and not so simple customizations. Not a problem. Time moves on. I used to program DOS screens in Quick Basic before anyone had ever heard of Windows. I understand and welcome new challenges even at my age.
I have not yet clicked the link to read how its put together yet, but I will as soon as I finish writing this. Up until now most of this "how" has been limited to the internal development staff there at xTuple I don't mean to imply it was a secret, there just hasn't been much of an effort to educate the reseller community like has been done in the past with scripting. I guess I am looking for assurance that kind of education and training will be forthcoming.
I think we (xTuple and its entire community) have been reluctant to actually say the desktop client would some day go away for fear of hurting sales. The idea has been "sell what we have now". It seems clear that what we have now is really nothing at all like what we will have someday, and maybe that someday is coming pretty soon.
So, I say all of this just ask if maybe it is now time to lay out exactly what to expect going forward?
Mon, 05/12/2014 - 10:18#2
Larry, I hear what you're
Larry, I hear what you're saying about how our next-generation platform has, until now, felt like a secret to the larger community. It was never our intent, of course, but the new technology has been *in the lab* for a long time. The breaking news, which we're all very excited about at xTuple, is that this product is now out of the lab, and we're implementing it around the world.
Getting our community up to speed with the new technology is one of our top goals here in Norfolk. There's already a lot of documentation in the link that John provided. Setting up a vagrant dev VM and working through the Ice Cream tutorial are the two things you'll probably want to start with. More training and documentation is on the way, particularly through our upcoming xTuple U initiative. And, of course, doing field dev in this next-generation platform is going to feature prominently in the training sessions of this year's xTupleCon. By the end of the year, it's my goal that you'll be sick of hearing from us.
You should also know that we're not racing to replace our desktop client with this new product. If we were, then our focus right now would be on re-implementing the functionality of every module of the Qt app, which we're not doing. Instead, we're spending our development resources to make the webapp shine in the areas that it's particularly suited towards, like mobile inventory, BI, and integrations.
Exciting times ahead,
Sat, 05/10/2014 - 18:29#3
Having invested a huge amount of time and effort in understanding the Qt Desktop client and scripting extensions, I too was cautious about the shift to the new technology platform. John's article above explains in a lot of detail the many, many reasons why xTuple made the difficult decision to migrate.
Having spent a lot of time on the new platform I can safely say that that it truly is leading edge and definitely a platform for the future. Whilst some things in the new technology require quite a shift in understanding, John's comments about object oriented programming are absolutely on the mark. There are so many things you can do with the Mobile Web platform that are just not possible with the Qt desktop. Furthermore the modular approach to the design, build, security and user experience is liberating.
xTuple customers are going to be able to achieve so much more functionality on this platform. This is a world-class platform for the Worlds #1 ERP system.
Sat, 05/10/2014 - 20:35#4
Just to put a little
Just to put a little definition on the above comments, for those who don't know all of this... Dave Anderson is the author of the Fixed Asset suite of extension packages for the Desktop client - three distinct packages for FA registry, depreciation, and maintenance. I think it's safe to say there is no more substantial third-party extension to the Desktop client in the xTuple universe.
(That's part of why we hired him ... so if you'd like to discount his comments as an xTuple employee there's your reason.)
But seriously, this is a guy who's spent a long time looking at the "old" and the "new" .... in fact, he's currently working on a major new extension to the mobile web platform, which will enable a fully-integrated Quality Management System into the Manufacturing Edition.
So, to speak to Larry's questions above, we absolutely want to put more information into everyone's hands, as fast as possible. I'd start, of course, with the links John included in his original post... but if, after reading through all that and fiddling with the code, people have specific questions, please post them here.
Also note that, of course, details on the new platform will be a major part of xTupleCon14.
Sun, 05/11/2014 - 07:30#5
I'm curious if there has been
I'm curious if there has been any discussion to develop a POS module with new platform. Would that be possible? Or would have to be an app more like the old QT platform?
Sun, 05/11/2014 - 08:44#6
A new POS solution is very much on our radar. The Sales Order entry screen is already vastly improved over the Desktop client (includes the #1 requested item, grid order entry)... and credit card integration is complete as well (much simpler than the sea of C++ required to do it in the Desktop). We're working on signature capture as well for some distribution customers.
Now obviously, a true POS would need to be much simpler than the full Sales Order screen, and there would likely be other additional features required. If anyone would be interested in collaborating on this, and/or sponsoring such an effort, please let us know.
Wed, 05/14/2014 - 19:34#7
I have one potential customer
I have one potential customer for xTuple later on this year and they will require a POS system. Not sure if they would be willing to sponsor this development but I will bring it up for sure.
Mon, 05/12/2014 - 08:13#8
Excellent post John!
Excellent post John! Not only on the benefits of the Web Client that we can leverage now but on the sound architecture for future development.
I’d like to emphasize the importance of easy system to system integration. In Gartner’s report “Develop a Strategic Road Map for Postmodern ERP in 2013 and Beyond” the authors assert that the do-it-all ERP suite is no longer tenable and is being deconstructed into “post-modern” ERP consisting of federated applications. Different functions could be provided on premise or delivered in cloud apps or by business process outsourcers. Successful ERP platforms of future will have architectures that enable integration.
The new architecture allowed us to integrate Tuple and Business Intelligence. The integration uses the same user interface and allows navigation and “drill-down” between operational and BI views providing a seamless user experience. A browser-based user interface is key as all modern BI technologies are browser based. And the Web Services support with the REST interface enables the exchange of information.
Tue, 05/13/2014 - 15:28#9
I have read in some the document links in John;s post about the idea of an "xTuple Server". It is unclear to me if this means a collection of software that can be installed on a Linux machine or whether this means a Linux virtual machine.
Wed, 05/14/2014 - 13:14#10
xTuple Server is software
The xTuple appliance is a preconfigured piece of hardware, purchased from xTuple, on which we install the xTuple Server software. When we refer to the "xTuple Server" we are talking only about the software.
Wed, 05/14/2014 - 12:49#11
In Line Editing
Don't mean to be argumentative, but I was doing in line editing in DOS applications back in 1989. I don't remember exactly how, but I can assure you it worked. Probably have some of them around here somewhere. So, I believe it would be more accurate to say that in line editing has been difficult if not impossible with the QT tools.
Wed, 05/14/2014 - 14:03#12
Thanks, Larry, for brining
Thanks, Larry, for bringing this point up because I'd like to take this comment as an opportunity to elaborate a bit more on how our current development path compares to some alternative paths we could have taken.
Qt actually supports inline editors. It also supports a model system necessary to back those things. The Purchase Order screen has an example where we implemented a hybrid system of Qt models around the current hand written SQL paradigm. The problem is going halfway leads to some awkwardness in the application, and if you've used the "Express Entry" tab on that screen you know what I'm talking about.
To be fair as well it's important to note Qt didn't support models until Qt 4, which came well after most of the current application was written. So the conundrum here was to implement an object oriented data system we were facing a re-write of most of the application no matter what. You can see as far back as 2009 I was talking about introducing a data model system into the Qt application:
Here's the problem: There's over 700 screens in the application today. You must understand that if we were to have gone down that road ALL THE CUSTOM SCRIPTS THAT HAVE EVER BEEN WRITTEN FOR THE C++ CLIENT WOULD HAVE BEEN BROKEN. Furthermore, we like to do things in an incremental and agile manner here. How do you go about making such a change in an incremental manner in a giant monolithic application? Can you imagine if in every release we had just changed a few screens at a time? Some scripts would break on some releases and others wouldn't. When you'd go about writing a script you'd have to write it in SQL statements in some screens, and use object models in others. I think that would have been a huge mess.
If we'd gone all the way in one pass it would have taken years of time in the lab with no real world field testing. That likely would have likely introduced hundreds of new bugs in an otherwise stable and functional application. How many of you out there have lived through a major upgrade on an ERP system moving from one technology platform to another? Horror stories abound. The worst part is, if you found your business was devastated by an upgrade of that magnitude, there is no way to turn back. Finally, we still would not have achieved one of our major goals which is to get the app. to run in a browser to eliminate client deployments.
This is why it seemed prudent to build an entirely new client and leave the existing one intact. The Qt client is stable. It's feature rich. It WORKS. Following the path we have, we are able to build the new technology incrementally without threatening the stability of anyone's business. For those who complain that all their customizations will have to rebuilt on the new technology if they want to use it, consider how much worse it would have been if we had rebuilt the Qt client from the inside out so you absolutely could not upgrade until everything was re-written. They way it works now, people can upgrade, play with the new web client and reproduce any customizations they need on it at a leisurely pace while still using the tried and true Desktop client.
Wed, 05/14/2014 - 14:53#13
Yes, John I understand everything you just said. I also understand you were trying to laud some of the virtues of the Web Client without having to go into as much detail as you just did about just one of these virtues.
I didn't mean to put you in a position to have to go into that much detail just to respond to my off hand remark. Sorry.
I guess I am just still a little put off that I cannot connect with the desktop client to a database on a Windows Postgres machine that has been created from a backup of a database off a Linux machine with the full Web client implemented.
Any chance something could change there?
Wed, 05/14/2014 - 16:35#14
Hi Larry, Nothing to
Nothing to apologize for. I just thought you brought up an interesting point about the limitations of Qt, or lack thereof, and that made a good launch point to describe some additional justifications about our chosen development strategy.
As far as running Postrges natively on Windows with the new tools goes, that's not really on the priority list. Technically it is possible of course, but community support for all the components of the stack that comprise xTuple Server on Windows is spotty. It sounds like you're saying you want to move a database that has been upgraded to the web environment back to Windows, and this is also possible, but is not in any kind of officially supported scenario.
One of the challenges we are facing, and have indeed faced for a long time, is this idea in the open source world that people can and should be able to do anything imaginable with the product. Run it on any operating system, change the code anywhere, and connect anything and everything to it by any means. The trouble with this thinking is it leads to implementations that are difficult, if not impossible, to support because there are no scope boundaries. I think what you are going to see at xTuple is an incremental shift toward tighter scope boundaries where we strongly encourage some practices and discourage others so that we can better service customers with a more standardized offering and best practices.
The bottom line is the database for the new server needs to run in a Linux environment. We are not precluding setting the system up in the "old" way on Windows, but the new tools require Linux. For people concerned about the administration of Linux the goal is to eliminate that concern completely by making xTuple Server essentially a black box that is administered through the xTuple application. Think of it like your smart phone. If you have an Android or Apple phone, you don't know or care how the nix based OS works. You just use the apps on the phone and even administrate it entirely through some of those apps. This is how xTuple Server is shaping up to work. In the long run such a device will be much, much easier to set up and manage than anything xTuple has previously produced.
Wed, 05/14/2014 - 17:36#15
I fear I am not making myself
I fear I am not making myself clear. I am not asking to use the new tools on windows. What I want to do is backup a database from a Linux machine that is fully configured to run the Web client, then restore that database to a Windows machine and connect to it from the desktop client.
It appears we are adding additional schemas, functions, and an extension that stops the desktop client from connecting when the database is restored to Windows. The culprit seems to be PLV8.
Just not clear to me why this has to be there for the desktop client to connect.
Wed, 05/14/2014 - 18:40#16
Quoting John: "It sounds like
"It sounds like you're saying you want to move a database that has been upgraded to the web environment back to Windows, and this is also possible, but is not in any kind of officially supported scenario."
Doesn't that answer your question? Possible, but not supported.
Wed, 05/14/2014 - 20:34#17
Thanks for joining in. Yes that is what I am trying to do. But, let me explain further why. You may or may not know I have been the first line support for an xTuple manufacturing client for about 3 years. That client has recently become an Enterprise customer and has the Web client installed and they are starting to use it.
As part of my support agreement with this client I back up their database weekly as an extra precaution to a machine here in my network because I have a VPN connection to the clients network. From time to time it has been very convenient to load this backup database on my server here in my office which happens to Windows. This to help troubleshoot an issue, verify a bug report, test a custom report or screen, test form printing, etc. Just very helpful to support this client. In fact I do that same thing for two other xTuple companies to which they have provided me remote access. Very early Sunday morning coffee, back up clients databases, eat breakfast, go play golf. Pretty good schedule.
Anyway in trying to research and/or duplicate a user complaint from the xTuple customer that has been configured for the Web client, I loaded the backup database on my windows server server to do some testing. I then discovered that I could not connect with the desktop client.
Sorry, for being so long winded, but I think it is important for you to know that I am not just wanting to do this because I want to. It is to perform support services for our client in the same manner I have done for three years. I do not need or want to connect to the database with the Web interface. Yet.
So then if it is possible, how do I do it?
If it is not possible with relative ease I will create a Linux machine configured with all the Web client stuff and I believe I should be able to create the client's database there to provide the same support I have been providing .
As I best I recall I have asked you more than once what it takes to run the Web client and your response has been "a lot". I know you wrote in one of the xTuple newsletters that it was "hard". Actually I saw those answers as very truthful, honest, and direct. I am a big fan of honest and direct. I have learned today that the development team is working on some tools so that it won't be quite so "hard". It is just that I wasn't prepared to do all the "hard" stuff to continue supporting a client as I have been doing just yet. But, if that is the way it is, I will figure out how to deal with it.
Thanks again for joining in. Great to see the boss listening.
Wed, 05/14/2014 - 23:02#18
The good news is, it's gotten a lot easier. From what you're describing, it sounds like the easiest thing for your use case of loading backups on your local machine would be to just install the VM on your Windows box.
I did it myself on my Windows laptop, live, in the middle of our recent quarterly all-hands meeting. We don't recommend a VM for production servers, but for what you're describing, it should be more than adequate.
Thu, 05/15/2014 - 15:11#19
Will I have to reinvent a lot of wheels?
Great write-up, John. While the discussion so far has been between xTuple employees and partners, I’d like to chime in with a customer’s perspective. Over the past three years, my company has made a tremendous investment of time, effort, and money into creating extensive and highly complex scripted and SQL-based customizations to the Qt client. These are not just reports that add to xTuple’s native capability to extract and present data. Instead, these are integral parts of our implementation and we could not successfully use xTuple without them. In our particular case, the bulk of our customizations have been made to the sales order process. We cannot use the mobile web client for anything besides basic data viewing without these customizations. However, the prospect of designing and developing them all over again using the tools described in the web client documentation (with which I have zero past experience), and incurring that expense of time, effort, and money all over again, is daunting and likely inconceivable. xTuple isn’t my only focus, and I doubt it’s the only focus of my I/T colleagues at other xTuple customers. When I have to build something into or onto xTuple, I want to do it once so I can move on to the other 500 things on my to-do list. I’m OK with the effort involved in making sure it works with xTuple upgrades, but I don’t want to have to do it all over again to fit a new operating environment. I would therefore suggest that xTuple investigate the possibility of creating a tool that migrates and converts scripted custom modules into the format needed to run on the mobile web client. I don’t know if this is even remotely possible to do, but it would be great for customers, and great for xTuple, too. Without something like this I fear that existing customers who have built a lot of customizations will not adopt the mobile web client at the level you would like, and those customers may get left out in the cold as the company’s development focus moves steadily away from the desktop client.
Fri, 05/16/2014 - 15:00#20
Hi Keith, your request is a reasonable one, and I see where you're coming from. The ability to write your own customizations on top of our core code is part of what makes open-source great. The customizations that you've made to the database (functions, etc.) will likely work just as well with the new platform; they share the same database code, after all. Unfortunately, there's just no way to automatically port over the customization that you've written to change the behavior of screens on the Qt client. We'd do it if we could, but it's a technical impossibility.
It might not be so hard as you're fearing to rewrite your custom code. Even though the presentation of the screens is vastly reworked, the business objects still behave the same way, so you won't need to re-grok the logic of our objects. Still, it will be a major undertaking if you have a lot of custom code. And, of course, it will require the investment in learning how the new product can be customized.
To that end, we're making an A/B comparison of some classic customizations a major feature of this year's xTupleCon. For those of you who will be there, this will be a great opportunity to understand the methodology differences you'll need to know. For those who can't be there, we'll be publishing plenty of reference material towards this end which should serve as a good starting-off point.
We absolutely understand how much value our community adds to our product, and are fully committed to get everyone up to speed with our latest developments as fast as we can. The transition will, of course, sometimes be a difficult process, as it has been internally here. But we think that everyone is going to be thrilled with the end result.