jQuery Performance: Problem & Solution

One of my current tasks is to refactor our SharePoint portal’s design.

The original design was done with heavily laden with tables and large images.And each new subsite had its own master page, style sheet, and images directory. It was performing poorly in the browsers, and making it painful to create new subsites.

So I’m making a CSS-based version, removing all the images, simplifying the design while improving performance and flexibility.

The end result was much faster, but the look was too simple. It needed just a little something to give the design some depth.

And that is where jQuery came in — I picked up a Drop Shadow plug-in, and put some shadows around the main page elements. It was just enough to make the page look pretty decent.

BUT — performance degraded exponentially every time I worked with web parts. I haven’t determined exactly where or why, but I do know that somehow the jQuery calls combined with what SharePoint does to a web part page in edit mode was very unhappy. As-in, browsers hang and crashes almost every time I worked with a page.

So, I turned off the jQuery when a web part page is in edit mode by just popping the following into the code:

if ($(“.ms-WPAddButton”).length == 0){
…Do the jQuery Stuff…
}

In other words, I use jQuery to look for the ‘Add a Web Part’ buttons. If they are not found, I continue with the script. Editable pages have the simple interface, and run quickly. Normal content pages have the nicer interface… and still run quickly.

There may be a better way to handle this, but it has worked for me so far.

Notes Dev Hall of Shame #2

The main topic for today is user-configurable data.

It is true that hard-coding data is bad. The values in various drop-down lists, etc, would ideally be populated from data that the business owners can update.

But somehow, the idea of “hard-coding is bad” got misunderstood. One former developer obviously missed the point… that the idea was to not require code changes when data changed.

So we have a database full of hundreds of configuration documents. When you put these documents into edit mode, a big red warning pops up. “Do NOT change these documents without checking with Notes Development staff — code changes will be required!!”

Ehh… what??

So I look at the code. While it is true that these documents populate drop-down lists, the agents that then process documents are full of hard-coded business logic…

“If docmuent.FieldA(0) = “Data from config doc” Then…”

So if the data changes, the agents all break. So not only does any data change require a code change, but we are actually inviting the business to break things, and doing so in a way we need to debug code instead of just updating a field.

And this is done all through the entire Notes platform. To the point that the business community here think that these kinds of configuration documents are a built-in feature of Notes, and all Notes apps will always work this way.

In an environment like this, is it any wonder that people think Notes is a hopeless platform? I have a strong suspicion that if the Notes system had been well built from the start, SharePoint would never even have been a consideration here.

Road map for functions in the SharePoint World

When people say “SharePoint”, they usually mean any application accessed through a SharePoint portal. In truth, this may actually be SharePoint, but could also mean .NET or InfoPath applications.

One of the challenges we’ve had is defining which types of applications really are a SharePoint app vs. InfoPath vs. .NET.

But before I go there… why does it matter?

It matters in our organization because of the various roles certain individuals play. Power users and business analysts can do some of their own work… if it is truly pure SharePoint. We’re finding that Domino developers pick up InfoPath so well, they can do more with it than many Microsoft experts. (Which is a blog topic in and of itself, but in short, we’ve spent years working with form-based platforms… we know what to do with, for example, Conditional Formatting (Hide-Whens) to simplify forms instead of making new forms and views for minor UI differences.) And… .NET — while our team can certainly write the code, it brings on whole new levels of deployment and maintenance issues. So we try to avoid it.

Which brings us to the “Road Map”. Which is not very profound… it is a very simple thought exercise for any application function that we are asked to provide:

1) Can SharePoint do it? Then make it so.

2) Well, then, can InfoPath do it? Then make it so.

3) Well, then, we’ll have to write .NET code.

After some discussions, we also added a question 0:

Does this even need server-side code? If JavaScript in the browser can do it, don’t get too clever with SharePoint… just write the script.

We had to add this question because developers who have never really been web-focused are forgetting that SharePoint ultimately renders in a web browser. Throwing some script in a master page or even in a content editor web part is sometimes a very quick, easy solution to some problems.

Our most recent example of this was a ‘Contact Us’ link. It was supposed to be available on all pages, provide an infopath form, then return the user to their original page on submit. They first wrote .NET code to do this. I talked them into just using a SharePoint list. Then they were doing some crazy SharePoint CAML stuff to make it happen. I talked them into just using JavaScript to append ‘&Source=originalpageURL’ to the HTML link.

I use this as an example to illustrate the eternal point of KISS — if 1 line of JavaScript code does the trick, don’t go writing .NET features. Find the simplest solution.

Notes Dev Hall of Shame – #1

I’ll pull actual code samples for later installments of our Hall of Shame, but first I simply wanted to share a few choice items that have not only boggled our minds, but also caused endless amounts of helpdesk tickets.

1) The EmployeeID field in names.nsf

At first glance, it almost looks reasonable. They added a field to the person document, labelled “Employee ID”, called EmployeeID, and updated nightly from our HR system. All applications look to this field when they need to get user data, as it is the common data point between every platform in our enterprise.

Or is it??

As we started to get all kinds of calls that applications were breaking, we found that about 20% of the applications read this field. The other 80% read ‘EmplID’. Also in the person doc. NOT displayed in the UI. Nor was there and agent to set it.
Our old admin team set this field manually when they created new users, using agent local to their system. Only when we switched to a new admin team did the problem crop up.The new admin team was already setting the visible field, and we ended up hacking together a little agent to keep both EmployeeID fields in sync.

2) The Action bar of Doom and Despair

Let’s ignore for one second that this action bar has 125 actions, and all of them are the same code, copied and pasted with minor modifications.

Let’s even ignore that any given user only will see 2-3 of these actions, based on their role within the system, so there may have been a simpler way to do these functions.

Lets just focus on the hide-whens. The hide-whens that repeat the same DBLookup 5 times in each formula. Repeated across 125 actions. With some actions having 15-20 DBLookups, instead of the paltry 5.

Do the math — I counted 724 @DBLookups in this action bar just to determine which actions should display.

People wouldn’t even open the form. When I took this job, I was told that it takes 10 minutes to open that form, so they installed a Citrix system… because via Citrix, it is much faster. Really, is that the answer to an application coded so badly that it is unusable? To install Citrix to avoid the network bottleneck?

I rewrote all the code into a single function, wrote an agent that does all the lookups in a batch process each night, and sets your personal access in a profile document. One lookup to the profile document to know which actions to display. One set of actions with dynamic labels and a function call. I was left with 10 actions, 1 function, 1 agent, and 1 lookup. And the form now opens in 3 seconds.

Excel Password Removal Tip

So this has nothing to do with Domino nor SharePoint, but it is something nifty that I thought I’d share:

I had a need to modify an Excel form sent to me by a vendor. The form was password protected, so I did some quick searches to find tools to remove the password. To be honest, I didn’t trust the tools that I found.

So i tried the following procedure. It works great:

1) Open your Excel file in OpenOffice

2) Re-save it as an Excel file.

3) Re-open in Excel and now you can remove protection without being prompted for a password.

Thanks, OpenOffice!

Another Way to do InfoPath Tabbed Tables

Tabbed tables are a fairly common user interface elements, so I was suprised that InfoPath does not include them as a built-in component.

There are plenty of examples on the web of how to use views and buttons to create tabbed tables… but I felt they were all flawed for one basic reason — it limits your UI to one single tabbed table, with nothing nested.

So I whipped up a sample of how to create multiple tabbed tables on one form, in one view. This allows each table to be navigated independently of the others, and also allows for either ensted tables, or for horizontal and vertical tabs to be used to navigate an entire table grid, displaying one cell at a time.

Examples available in the download: Download Sample Form
(rename to .xsn…. wordpress doesn’t support .xsn uploads…)

InfoPath equivalent to @UserName

So in Notes, popping in a Names field with a default value of @UserName is a very simple, easy action.

In InfoPath? Not so much.

I first had to seek out how to even get a name selection control running in an InfoPath form, and then I discovered that the XML data needed for that control made a simple userName() call ineffective. A plain-text field can easily pull the current user, but not a selection control.

So I spent some google time compiling solutions, and found how to make the control pull the current username. Cool. But the answers I found pulled the name on all form loads, not just the initial load. So I had to get the rule to run only on the first load. Doing field comparisons to only run if my values were blank didn’t work. XDocument.IsNew was not recognized as a valid formula. So I ended up doing another hack with inappropriate XML data to flag what status the document was in.

I’d like a cleaner answer to running a Form rule on initial load only — does anyone have one?

In the meantime, here is the ugly detail that I emailed to my co-workers in case they need to do the same time:

If you want to add a field to an InfoPath form with a name selection from AD, follow instructions here:

http://blogs.msdn.com/infopath/archive/2007/02/28/using-the-contact-selector-control.aspx

These instructions are very simple compared to what it takes to make the current user default into this control.

To do that…

1) Set the required default values:

• From your Data Source Window:
• Navigate to your XML data structure that you build for the contact selector.
• Right-click “DisplayName”
• Click the “fx” button next to the default value field
• Enter “userName()” Case sensitive, and you have to enter it via this dialog box, otherwise it is treated as text, not as a function.
• Also add a new field to your data source, called “IsNew”, text, default value = “1″ (This will be used to be sure the data is only set on initial form load. I tried LOTS of cleaner methods, but this is the only one I got to work reliably.)

2) Make a connection to the SharePoint Web Service to check names against AD:

• Select “Manage Data Connections”
• Click “Add”
• Click “Create A new connection to:”, Select “Receive Data”, Click “Next”
• Click “Web Service”, Click “Next”
• Enter: “https://your.sharepoint.server/_vti_bin/people.asmx?WSDL”, Click “Next”
• Select “ResolvePrincipals”, Click “Next”
• You will see a list of the data for the Web Service — just click “Next”
• Uncheck “Store a copy of the data in the form template”, Click “Next”
• Uncheck “Automatically retrieve data when form is opened”, Click “Finish”
• Close The Data Connection dialog box.

3) Open the Form options to set up a query to this web service upon open

• From the main infopath menus, Select Tools –> Form Options
• Go to the ‘Open and Save’ page\
• Click “Rules”
• Click “Add”

4) Set the default value from your Display Name into the Web Service Query

• Click “Add Action…”
• Select “Set a fields Value”
• Click the box next to the “Field:” field
• Change the Data Source Drop Down to “ResolvePrincipals (Secondary)”
• Navigate the tree to select “myFields –> queryFields –> tns:ResolvePrincipals –> principalKeys –> string”
• Click “OK”
• Click the “fx” box next to the “Value:” Field
• Click “Insert Field or Group:”
• Navigate to the XML structure you built for your contact selector – select the DisplayName field
• Click “OK”, Click “OK”

5) Set the principalType fields for the Web Service Query

• Click “Add Action…”
• Select “Set a fields Value”
• Click the box next to the “Field:” field
• Change the Data Source Drop Down to “ResolvePrincipals (Secondary)”
• Navigate the tree to select “myFields –> queryFields –> tns:ResolvePrincipals –> principalType”
• Click “OK”
• In the “Value:” Field, enter: “User SecurityGroup SharePointGroup DistributionList”
• Click “OK”

6) Tell the form to perform the query

• Click “Add Action…”
• Select “Query using a data connection”
• Under Data connection, select “ResolvePrincipals”
• Click “OK”

7) Set your DisplayName value from the Query Results:

• Click “Add Action…”
• Select “Set a fields Value”
• Click the box next to the “Field:” field
• Navigate to your contact XML, and select “DisplayName”
• Click the “fx” box next to the “Value:” field
• Click “Insert Field or Group”
• Change the data source to “ResolvePrincipals (Secondary)”
• Navigate to and Select “myFields –> dataFields –> tns:ResolvePrincipalsResponse –> ResolvePrincipalsResult –> PrincipalInfo –> DisplayName”
• Click “OK”, Click “OK”, Click “OK”

Set the accountID value from the Query Results:

• Click “Add Action…”
• Select “Set a fields Value”
• Click the box next to the “Field:” field
• Navigate to your contact XML, and select “AccountID”
• Click the “fx” box next to the “Value:” field
• Click “Insert Field or Group”
• Change the data source to “ResolvePrincipals (Secondary)”
• Navigate to and Select “myFields –> dataFields –> tns:ResolvePrincipalsResponse –> ResolvePrincipalsResult –> PrincipalInfo –> AccountName”
• Click “OK”, Click “OK”, Click “OK”

9) Set the AccountType value from the Query Results and close the dialog boxes

• Click “Add Action…”
• Select “Set a fields Value”
• Click the box next to the “Field:” field
• Navigate to your contact XML, and select “AccountType”
• Click the “fx” box next to the “Value:” field
• Click “Insert Field or Group”
• Change the data source to “ResolvePrincipals (Secondary)”
• Navigate to and Select “myFields –> dataFields –> tns:ResolvePrincipalsResponse –> ResolvePrincipalsResult –> PrincipalInfo –> PrincipalType”
• Click “OK”, Click “OK”, Click “OK“

10) Set the Rule to only run on the initial load of the form

• Click “Set Condition…”
• In the first dropdown box, select “Select a Field or group”, then navigate to your “IsNew” field that you created earlier.
• 2nd drop-down = “is equal to”
• 3rd drop-down, select “Type Text…”, and enter “1″
• Click “Add Action…”
• Select “Set a fields Value”
• Navigate to your “IsNew” field for the “Field: field.
• Value: 0
• Click “OK”, Click “OK”, Click “OK”, Click “OK” (Close out all dialog boxes)

Whew! Now you are done and it should work..

———————

Just as a comparison, In Notes:

• Make a name fields
• Set the default value to @Username
• 

Fun with InfoPath

I’m having fun this week. I’m digging into InfoPath for the first time, as we’re now into some of the more complex forms that really are not readily handled by basic SharePoint lists.

I’ve discovered a few tidbits that I wanted to document here. Please note that I’m a flippin’ newbie when it comes to InfoPath, so some of these notes might be obvious to others, and some of them might be bad ideas and show my inexperience.

InfoPath is an XML-based product. Underlying your forms is (normally) an XML schema. This actually translates very well to Notes data. Notes parent/child relationships between documents mesh very nicely with an XML scheme that includes child document data as a node in the XML.

What this means is that I’ve been able to take multi-form Notes databases, and replace them with a single InfoPath form, published to a SharePoint library.

However, this approach did have a few challenges:

1) User Interface

Putting all that data on one form is a UI challenge. In the two forms I’ve worked on this week, I’ve used tabbed tables. One is your typical tabbed table, with tabs across the top. The other was a very long form (about 30 pages printed), so I created vertical tabs down the left side of the page that act as a table of contents for the form.

There are articles out there on how to do tabbed tables in InfoPath. Go read them to get the idea of what controls you need to lay out, then come back here, because I did it a more efficient way. Instead of using your buttons to switch to a new view, think like a Notes Developer — Use “Hide-Whens”. In InfoPath, it is called “Conditional Formatting”. Create data fields to store which sections should show, and have your buttons set those fields appropriately. Then just set your conditional formatting off those fields, and you have tabbed tables in a single view.

This also has an added benefit that you can create multiple tabbed tables that all operate independently of each other. The view-based solutions online do not allow for this.

“But wait — isn’t that mixing UI data into your XML? That is a horrible thing to do.”

Yes. Yes, it is. First off, relax. We’re already on Microsoft products, so whatever you do is a hack, no matter how clean you make it.   But seriously, I did make an attempt to address this issue. I created a new node in my XML to hold this form metadata. If I need cleaner XML to send this data to another system, I just need to drop that node. And in the meantime, my UI holds its state between form loads. XML purists may hate my solution, but XML purists probably aren’t corporate IT grunts doing a job like this anyway.

2) Security

We don’t have authors fields. We don’t have readers fields. We need to make SharePoint security do everything for us, or use pseudo-security via the UI. For SharePoint security, we’re kinda out of luck for Readers/Authors functions, but it can handle most other security needs. For UI security, it isn’t truly secure, but depending on the application needs, it might be secure enough. InfoPath and SharePoint are not like Notes, where a savvy user can edit the document data by making up their own forms and agents, etc. You cannot right-click and see data fields, bypassing the UI. Even for most savvy users, they have to use your UI. So if you, for example, disable all controls in your InfoPath form unless the current user is listed in a specific field, that will actually work more reliably than a similar technique in Notes.  There are still ways around it, but it should be enough of a deterrence to be effective for basic business apps, especially if you have revision history turned on in the SharePoint Library and can throw out any inappropriate edits to your documents.

3) Workflow

InfoPath coding cannot send email notifications. But it does very well at creating buttons that perform different actions. SharePoint workflows are more complex, if they need to perform logic on your InfoPath data, but they can create workflow tasks and emails very nicely. I haven’t yet got all the answers as to what the best practices will be when making choices between InfoPath rules and SharePoint workflows.

For the form I finished today, I did most of the work in InfoPath, then created a SharePoint workflow to handle notifications for the approval cycle.

To make your InfoPath fields available to SharePoint, you must select them when publishing your form. It is hard to miss this step, as the publishing wizard asks you very directly which fields you want to expose to SharePoint. Be sure to expose anything that your workflow will want to read.

If your workflow also needs to set an InfoPath field, there is a checkbox that reads “Allow users to edit this data in a datasheet”, or something to that effect. Select this checkbox. If it is not selected, you cannot set the field with a SharePoint workflow.

If you have multiple fields in your XML data source with the same name, but under different nodes, SharePoint pulls them all in with just the name, and loses your hierarchy. I didn’t delve deeply into how much this would screw things up, I just got a sense of ugliness and decided that I needed a different way to handle things. So I made copies of the data I needed in my meta-node that I used for UI data, and I exposed these copies instead. Is this a bad idea? I don’t know. There may be a cleaner way around this particular problem, but I have not yet learned it.

————————–

In general InfoPath is a nice tool. It will feel more like Notes Form development to us Lotus folk, and provide many of the features we are used to. Much of our design skill and UI experience will translate directly across, and I wouldn’t be surprised if Notes people start becoming very innovative in InfoPath, as we think differently than .NET folks, so we will approach form development with new ideas. I’m looking forward to getting past this phase of learning, and seeing what I can really do with the tools over the next few months.

The fun part of migration efforts

There is one part of migrating to a new platform that I am truly enjoying — the cleanup work on the Notes/Domino system. I love taking an old, cumbersome app, and refactoring the code into a quick, simple design with significant performance and UI improvements. I can usually reduce the support needs at the same time, frequently reducing the headcount required to support the apps by well over 50%.

We end up doing this kind of refactoring on many apps before thinking about a migration because many of the apps have have devolved into such massive heaps of spaghetti code tht they need a good refactoring just to understand exactly what the current functions are.  Once the refactoring is done, then we talk to the business about whether all components are still needed, and look at whether or not a change in a business process may allow us to simplify the app even more. What we find is that through this simplification process, the migration becomes only partially technical, but as often as not, a business transformation is the result of the technical work. A a chunk of the time, the resulting app is so easy to use and maintain that we just stick with it, and don’t migrate it after all.

Which leads me into my next train of thought – if the refactoring is what I enjoy, and what I excel at, why do I spend so much of my time on the areas that I do not?

So I am considering taking on side projects from other organizations. Seeking out other people, whether or not they are migrating, would would have an interest in hiring me to refactor their “ugly code”. For now, I am thinking just 5-10 hours a week on the side, unless someone has a project so major that it makes it worth switching to an independent consultant on a full-time basis.

So what do you all think? Is that something you would have any interested in contracting out?

Correcting My Ignorance

A year ago on this blog, I listed some Notes functions that did not have equivalents in SharePoint.

Over this past year, as I learned more, I learned that I was just plain wrong.

So let me take the time to correct my statements,  and lay out the functions in SharePoint to replace the following Notes functions:

• Hide-Whens – Can be computed in your XSLT via SharePoint designer.
• Reader/Author access computed based on document data – Still not available based on my current knowledge.
• Flexible Workflow, sending to different recipients, or selecting a different workflow based on document data – Not available as a single workflow, but multiple workflows can be combined to achieve these types of functions.
• More than 2000 items in a view – Available, but not recommended. Apparently that limitation is not a technical limit, just a performance guideline.
• Private Views – Yeah, these exist. Stored on the server, though, not locally.
• Agents – Not available as a SharePoint function, but you can write scheduled jobs on the server in .NET, etc.
• Action Bar/Shared Actions -Available in Infopath.

There ya go – just reinforces my point that even the most open minded Notes person will not “get” SharePoint development right off the bat. It take times to learn the new moving parts and figure out how to put them together…