Offline SharePoint

Below are the questions participants asked at the recent Webinar “How To Implement an Effective Email Attachment Solution in SharePoint”. If you would like to view the webinar, please go here.

Q: If the SharePoint site is a hosted site and is authenticated via forms based authentication how will this affect the Outlook integration?

A: While Colligo does support Hosted SharePoint sites, at this time it does not support Forms based authentication. This is a feature we are actively looking in our roadmap for a future release of Contributor.

Q: When we work offline, where are the offline docs stored? Is it in the local .pst files?

A: Colligo Contributor does not store data in a .pst file. It stores information and the local cache outside of regular Outlook files and therefore minimizes any impact to Outlook. The SharePoint structure and metadata is stored in a SQL database and the SharePoint content is stored, in native file format, in the Windows file system.

The location of the cache is configurable but by default is located in the local users’ application data area e.g. c:\windows\users\USERNAME\Appdata\Local\ColligoOfflineClient in Microsoft Vista. The cache is shared and used by all of the Contributor interfaces – Client, Outlook Add-In and File Manager.

Q: Is there full support for Outlook 2003?

A: Yes, Contributor supports the use of both Office 2003 and Office 2007 – and the Outlook Attachment Handler feature is available for both Outlook versions. In addition, Contributor is supported on both Microsoft XP and Microsoft Vista installations and 4.0 will provide Windows 7 compatibility.

Q: How do you upload incoming email attachments directly to SharePoint WITHOUT saving to hard drive first?

A: The Colligo Contributor attachment handler is designed to automatically upload outgoing email attachments to SharePoint then replace them with a link. Attachments in incoming emails can be uploaded using a simple drag and drop. For example, using the Contributor Outlook Add-In, the destination SharePoint locations are revealed as Outlook folders in the users’ mailbox area. Users then simply drag attachments from an incoming email and drop them onto the desired folder. Contributor will upload the item to SharePoint. In fact, the complete email can be similarly moved and Contributor will create an .msg file including the attachments, auto-extract the email metadata and then upload it to the SharePoint site.

When users link SharePoint document libraries into the Outlook folder tree, they can choose whether the library is to be cached or not. If the library is cached, then all emails are saved to the local hard drive then synced to SharePoint. If the library is not cached, the emails are synced directly to SharePoint (though they are cached temporarily to enable offline operation), and are not saved on the hard drive.

Q: Can you only use Contributor for internal e-mails within a company or only those who have access to the SharePoint site?

A: No – the Contributor Outlook Attachment Handler facilitates the uploading of attachments to SharePoint without placing any restrictions or limitations on the SharePoint site. It does, however, require that the recipients have access to the SharePoint location or locations where the attachments are stored.

In addition, due to the wide variety of email policies, it was determined that the ability for enterprises to create custom rules engines to match their particular email business logic was the best extension to our solution. This enables enterprises to determine what they want to happen in any scenario and then use the rules engine to assist their email users in meeting those requirements.

Q: What about the email itself? Does this get stored in the library or just the attachment(s)?

A: The Colligo Outlook Add-In Attachment manager facilitates the uploading of attachments or the insertion of existing content as attachments into draft emails. Because the email is in draft form (it has not actually been sent through Exchange) the feature does not currently upload a complete .msg file.

The Colligo Contributor Add-In for Outlook, however, does support the uploading of complete emails as .msg files to SharePoint via a drag and drop to Outlook folders that are linked to SharePoint document libraries.

Q: Has any of this changed with SharePoint 2010 and SharePoint WorkSpace 2010?

A: Unfortunately until Microsoft publishes the final feature set of the 2010 products, we cannot comment about specific features or uses. However, as a Microsoft Gold Partner, Colligo works to align its products with the Microsoft roadmap and to extend their ease of use and functionality to meet current business needs. Contributor supports its feature set across a range of Microsoft product versions enabling customers to migrate freely while maintaining a consistent user experience.

Q: How do you handle security for the attachment location on SharePoint?

A: The Contributor Outlook Add-In Attachment Handler facilitates the uploading of attachments to SharePoint without placing any restrictions or limitations on the SharePoint site. In order to open and view attachments, it does require that the recipients have access to the SharePoint location or locations where the attachments are stored. A custom attachment rules engine may be developed to ensure such access permission exist for each recipient.

Q: How do you handle OWA users?

A: Contributor is a PC client application which uses the full Outlook email client. At this time the Attachment Handler feature is not available via the OWA client.

Q: How much of the rules code is already written? In other words, do I have to be a coder to use this?

A: No, you don’t have to be a coder. Contributor provides an out-of-the-box attachment rules engine which manages uploading based on limiting the total size of included attachments. This has configurable limits for both warning and blocking emails from being sent and requires no coding. Partners are available to develop customized engines if required.

Q: Can you use the plug in to upload received files or to force attachments to be stored in SharePoint?

A: The Contributor Outlook Add-In can be used to manage inbox content. The standard features in this product include ability to drag and drop emails and/or their attachments to SharePoint locations which are identified in the users Outlook folder tree. In addition the Add-in support the use of Outlook rules to provide a more automated method of moving emails into SharePoint. More information on using SharePoint to manage emails can be found in our Resource Centre.

The attachment manager in Colligo Contributor 4.0 can be used to force users to upload attachments to SharePoint.

Q: What if I am travelling and I get an email from a co-worker whose attachment is uploaded to SharePoint? Am I forced to connect using VPN to access SharePoint and thus the attachment?

A: Contributor will insert a URL to the storage location within SharePoint and the recipient will need to access the SharePoint site to examine the attachment. The actual connection to the SharePoint site will depend on the capabilities of the device used to receive the email and where and when such access is required. For example: this could be used to support corporate information policies where emails can be read on mobile devices but documents must be secured with the enterprise domain.

If you are using Contributor to cache the location where email attachments are stored (such as a team or department site), you would be able to access it when you are travelling (assuming the email was sent prior to your last sync).

Q: I want to restrict the reading of the attachment to just the recipients plus myself. Do I need to create document library and set the permissions accordingly to mail recipient list?

A: You could do that if you want. If the scenario is one where you are sending attachments to the same team members often, then you could set up the permissions beforehand. However, creating a document library per set of attachments may be too great an overhead if the recipients are constantly changing. Unfortunately we cannot enable users to set item level permissions through Contributor, since they are not directly available using client side APIs. A custom engine which combines the client side interface with a custom server side web service to set permissions could be a possibility. (This may have security and credential implications).

Q: Would there be any limitations imposed by my IT group that would limit the use of Contributor?

A: Colligo Contributor is a standard .Net application and is a client PC only installation which requires no server components or configuration. Generally if users are able to install software on their PC they can install and use Contributor. Contributor also uses the same SharePoint services as Microsoft’s products (Internet Explorer, Office, etc.) and it only extends whatever access has been configured on the SharePoint sites (i.e. it uses the same access and security configuration as the SharePoint site) so PCs which can access SharePoint are normally configured with sufficient access for Contributor. There may, however, be IT policies on information management, retention and storage, etc. which will impact the usability of Contributor.

Q: How does this work with SharePoint single sign on?

A: Contributor supports Active Directory credentials including NTLM and Kerberos but not Single Sign On at this time.

Q: When using the insert attachment link, how to search for content? Large lists should have search facilities (by doc type and wildcard)

A: The Attachment Handler is used in conjunction with the Outlook Add-in product which support filters and views to manage content and information. Your suggestion, however, for search type facilities will be added to our roadmap process.

Q: Any plans for Colligo to allow users to create rules to strip attachments from incoming mails and put them into the “Colligo for Sharepoint” folder based on set criteria (PDF from x address goes to this folder)?

A: At this time we have not formulated a specific plan for this type of feature although it is an item in our overall roadmap list.

Q: How is this licensed?

A: As a client application, Contributor is licensed on a similar basis to a Client Access License (CAL). Each user needs a license which is associated with the machine the software is installed on. Volume discounts are available.

Q: How can users quickly find the correct document library – if they are used to browsing the web front end and using bread crumbs?

A: In order to pick a document library destination for attachments, a user must first link it first to Colligo Contributor. They can do this from the browser (using the Connect to Outlook button on the tool bar) or by pressing the “connect” button in Outlook. Available document libraries are then shown in the list of locations when saving attachments in SharePoint. Contributor always remembers the last location picked and displays it as a favourite location.

Q: When will Colligo 4 be out?

A: Colligo Contributor 4 for Windows Vista & Windows XP is in the final stages of its beta program and is expected to be released the week of October 12.

Q: We use CRM 4.0, where e-mails and contacts can be archived. How does that fit that with Colligo and mails in SharePoint?

A: CRM has its own database of emails, contacts and information relating to customer engagement. There is no direct linkage of information between information stored in CRM and information stored in SharePoint. The decision whether information is stored in CRM, SharePoint or both would be a matter for significant IT analysis and would require a solution beyond the scope of Contributor, although Contributor could be part of that solution.

A recent post by Rees Morrison in the “Law Department Management” blog discusses how National Grid, a large power utility headquartered in the UK, is using SharePoint for document management.

I was pleased to see the following quote from Adam Davidson of National Grid:

We are also about to start deploying (after a successful pilot) the Colligo Outlook Add-in to give us an even better tool to upload emails into SharePoint, as well as allowing lawyers offline access to their matter files within SharePoint.

This really highlights how the Contributor Add-In for Outlook can help to reduce the barriers to adoption of SharePoint for content management applications, such as legal matter management. I was especially proud since we were selected by a team that has been recognized for it’s leading edge application of SharePoint. National Grid won the “In-House Legal Department Initiative of the Year Award” at the Legal Technology Awards in 2006 for their use of SharePoint for DM.

Thanks for the mention, Adam!

Rees mentions SharePoint in an earlier post from his blog and references an article in the ACC Journal by Nanci Tucker of the Simpson Neely Group, a provider of law technology consulting services. The article presumably evaluates the strengths and weaknesses of SharePoint for legal departments, though I couldn’t access the article from the ACC Docket archive since I’m not a subscriber.

It’s great to see SharePoint getting the notice it deserves from the legal community. As Adam from National Grid said:

…[SharePoint is] not as fully featured as some proprietary DM/PM [document management/project management] systems, but it cost less and is much easier for us to administer…

I have a feeling that many of the feature deltas will be closed in SharePoint v.next.

This is my sixth and last post in a guest series I’m doing here on “The Myths & Truths of Email Management with SharePoint.”. My last post was on SharePoint list scalability.

SharePoint is a great platform for managing email and attachments and has several advantages in the right scenarios. In addition to providing the capability to store, organize, and search for content, SharePoint enables email to become part of the content that is shared throughout the organization. This improves collaboration and content re-use. There are a number of alternatives for moving emails to SharePoint, including out-of-the box methods such as email-enabled lists, managed folders, and third party applications such as Colligo Contributor Add-In for Outlook. The key to success is building an architecture that is scalable, while making it easy for information workers to use.

PST Files – While some of the capabilities of Exchange and SharePoint overlap, there are distinct differences and advantages of one platform over the other for different scenarios. When important files are stored on individual PST files throughout the organization, the information becomes inaccessible islands. Worse still, PST files pose security and data loss concerns, and often lead to the loss of enterprise content. In reality, PSTs are a thing of the past. Storage and sharing of email and attachments in SharePoint offers distinct advantages for collaboration, document management, and discoverability through a rich search interface.

Exchange Public Folders – While I don’t recommend deploying Public Folders for new applications, you needn’t worry if you already have them. They will continue to be supported by Microsoft in the next major version of Exchange server after Exchange 2007, and for at least 10 years thereafter. For new Exchange deployments, SharePoint is “strongly encouraged” for application development.

If deploying both Exchange 2007 and SharePoint, it doesn’t make sense to build up a huge Public Folder deployment. It’s confusing to users and additional overhead for the team managing Exchange. Microsoft is making major investments in SharePoint and has indicated that it’s the preferred application platform for collaboration and document management going forward.

SharePoint Advantages – As discussed, SharePoint offers a number of advantages over PST files and Public Exchange folders for email management:

• It supports the sharing of email across the enterprise
• User-based email and attachment selection ensures that “important” content is stored
• SharePoint centralizes critical enterprise content on secure company servers
• Email becomes “structured” enterprise content when custom metadata is applied
• The email body and attachments become findable and reusable
• Information workers can take advantages of the version control and history features
• Content types support retention policies and regulatory compliance requirements
• SharePoint improves collaboration and enterprise content management

“Out of the box” there are two ways (highlighted in this series of posts) that Microsoft products support the integration of Exchange and SharePoint for email management: (1) “send to” email-enabled SharePoint lists and (2) “auto copy” Managed Folders to SharePoint.

Email-Enabled Lists – Email-enabled self service lists can easily get out of control and become a significant drain on IT resources. Microsoft IT decided against using email-enabled lists internally. Most organizations don’t need the email-enabled functionality and the oversight it requires, and users prefer to access archived emails in .MSG format (rather than the .EML format created in SharePoint lists). With proper planning and design, email-enabled lists can be deployed successfully. However, Managed Folders are a better alternative.

Managed Folders – Introduced in Exchange 2007, Managed Folders provide administrators with an easy way for users to archive email. Managed Folders can be configured to “auto copy” emails added to them to SharePoint libraries. This incredibly insightful feature can significantly reduce mailbox sizes, while capturing the intended emails (in .MSG format) and attachments. Still, abuse must be avoided to ensure that SharePoint does not become a dumping ground. Managed Folders require a solid information architecture design, and trained administrators who understand scalability and how to manage it.

SharePoint Scalability – Storing emails and attachments in SharePoint can have great advantages for project and knowledge management in corporations, if they are secured in the proper context. However, SharePoint lists and libraries can quickly be bogged down with too much content if not carefully managed. The scalability of SharePoint lists is a major consideration and requires oversight, especially when using email-enabled lists. The default “All items” view can be particularly problematic.

Training and guidance may be required to change corporate culture so users understand where and how content should be stored. Training users to tag content with metadata will increase the usefulness and “findability” of documents. Folders, custom views, indexed columns, and a query-based design are best for large lists to ensure fast performance.

Third-Party Solutions – When deploying SharePoint for email management, it’s important to consider what you get out of the box with Microsoft, and what you get with a partner. Most gaps you find in SharePoint are filled by a very solid partner ecosystem. For example, Gold Certified Microsoft Partner Colligo Networks offers an Add-In for Outlook that makes it easy for users to drag-and-drop emails into SharePoint, while reducing the burden on IT. I would encourage you to try their .Net client-only solution to see if it can meet your Outlook and SharePoint integration requirements.

Well, that’s it for this series of posts on the OfflinesharePoint blog. I hope you found them interesting and informative. If you’d like to keep up with my latest posts, please visit my blog “SharePoint Joel’s SharePoint Land”.

- Joel Oleson.

As part of our continuing series on email management in SharePoint, I wanted to comment on Joel Oleson’s last post on storing all emails and attachments in a single document library. As he points out, SharePoint lists can exhibit performance problems when they are used to store large numbers of items. Since Colligo Contributor is a client-based solution for storing and displaying SharePoint lists, it can often improve list rendering performance for users.

Colligo Contributor will reflect server performance optimizations to the client since it supports folders, views, and filters. In addition, the performance of the Contributor interface is usually significantly better than the browser for content cached locally, for example when accessing content for a specific view. A unique feature in Contributor, which is not available out-of-the-box in SharePoint, is default folder-level metadata. This feature can provide advantages for tagging content since users can set default metadata values that get applied automatically when content is dragged into a SharePoint folder in Outlook. Colligo Contributor 3.2 supports large SharePoint folder hierarchies rendered through the Outlook interface.

Colligo Contributor has strong traction in the professional services market (accounting, construction, legal, management consulting, architecture, etc.) This is not surprising given our rich history with the Workgroup Edition product in the audit space. More recently we have been picking up momentum in the legal vertical with the Contributor Outlook Add-In and we have released a case study with Fischer Behar Chen Well Orion & Co.

As a result of this growing success, I am in the process of educating myself on the needs and careabouts specific to the legal market (i.e. trying to become a matter centric thinker).

As a starting point, I found an interesting link to an International Legal Technology Association (ILTA) survey here.

There were a number of interesting datapoints in the survey on the penetration of SharePoint, various “matter management systems”, eDiscovery, email policies, etc. so I wanted to pass it along for your edification.

This is the fifth in a guest series I’m doing here on “The Myths & Truths of Email Management with SharePoint.”. My last post was on Managed Folders.

Storing all emails and attachments in a single document library is a common practice and popular method for personal storage, however this is not a recommended best practice for knowledge repositories. In SharePoint, document libraries require special information architecture because of performance degradation associated with lists that contain a large number of items.

To gain a better understanding of the issues, there are a few documents that I would recommend developers and architects review, as outlined below.

• The whitepaper called “Working with large lists in Office SharePoint® Server 2007” has charts and metrics based on query time with performance considerations being a key indicator. The recommended limit of 2000 items per folder popularized in WSS 2.0 is not a bad one for those that don’t have the time to invest in designing a query based interface, though there is no performance “cliff” at 2000 items in a list. You may want to recommend to your users that they keep their lists under 1000 or use 3000 items as an upper limit at which point IT gets involved in assisting teams and groups to more efficiently scale their lists.
• Microsoft IT recently published a whitepaper titled “SharePoint Performance Optimization” that discusses ways of optimizing render time of sites and reducing database blocking. At Microsoft, the IT team has set-up a scan to detect large lists over 3000 items to ensure proper usage and divide up content where appropriate.
• Another useful article is the one on capacity boundaries on TechNet. It’s a good resource for seeing guidance on “limits” within the product along with charts on list performance.

If you currently have lists with a large number of items, or cannot avoid them in your application, below are a few quick ways to improve their performance.

1. Views can have a big impact on performance. The default “All Items” view should be avoided because it can take tens of seconds to render as a list grows into the tens of thousands. Views with too many items can can cause content database locking during query time. Changing the default view to something with a smaller number of items can often be a quick fix.
2. Filters are another way of limiting the number of items displayed in SharePoint. Note, however, these queries can be inefficient in large lists.
3. For document management deployments, use folders and sub-folders for better scale and retrieval. Keeping each folder with less than 2000 will support better performance. Even better, limit folders to one hundred or so to prevent unnecessary scrolling.
4. Using Indexed columns is another way to manage list scalability and increase index query optimization.
5. The most efficient method of retrieval is search queries.

Truth: SharePoint lists exhibit performance problems as list size grows, so it’s best to limit the number of items in lists and use views, folders, and filters to improve query performance.

Happy Valentines Day! In our continuing series on email management in SharePoint, I wanted to expand on Joel Oleson’s last post on Managed Folders. Here I will present an alternative to Managed Folders that let’s you store emails directly in SharePoint – Colligo Contributor Add-In for Outlook.

Colligo Contributor reduces the IT administration required to manage the routing of email content to SharePoint since it enables users to set-up their own links between Outlook folders and document libraries in SharePoint, based on existing user permissions set SharePoint. Like Managed Folders, Colligo Contributor automatically extracts email properties and moves emails and attachments to SharePoint when users drag-and-drop them into a folder in their mailboxes. In addition, Contributor enables the users to tag content with custom metadata and choose content type at the time emails and attachments are moved to SharePoint – a feature that is not available through Managed Folders. This can reduce the workload for legal site administrators and ensure that the proper retention policies are applied to content at the time it is saved by the user.

Happy New Year! This is my first post of 2009 on the Offline SharePoint blog and the fifth in a guest series I’m doing here on “The Myths & Truths of Email Management with SharePoint.”. My last post was on SharePoint email-enabled lists.

The subject of this post is Managed Folders. Managed Folders were introduced in Exchange 2007 to provide administrators with an easy way for users to archive email. Any Managed Folder can be configured such that all emails sent to it are routed to SharePoint. It’s an incredibly insightful feature and when implemented properly can reduce mailbox sizes, while capturing the intended emails and attachments. When not implemented properly, Managed Folders can be abused, causing SharePoint to become a dumping ground.

One poor example of managed folder design could be a managed folder called “Keep.” If everyone is told to put their email in that one folder and it’s archived to SharePoint, it sounds like a perfect solution for email archiving. However, there are some serious drawbacks. First, scale is an issue. Putting all that junk into one list can overwhelm SharePoint since it doesn’t scale well to support millions of items in one list, especially if there is a single view. Second, what about the security of that list? Managed Folders require extensive administrative set-up. It’s an IT option in Exchange, not a feature that is end user configurable.

So what does a good Managed Folder design look like? An example might be a folder titled “Legal Hold,” which is used to archive items under legal hold because of an investigation or other circumstance. On the SharePoint side, a specific document library is set-up and secured, then a legal site administrator is responsible for any tagging and for managing the views for the LCA team. A special search view might be set-up with specific indexed columns to support a quick and easy search. To avoid performance problems, avoid the “All items” view. I’ll discuss performance issues related to SharePoint list scalability next.

Truth: Managed Folders can work well, but require a solid information architecture design & trained administrators who understand how to manage scalability.

Happy New Year. We had a terrific 2008, thanks in part to the terrific growth of the SharePoint market. Let’s hope 2009 is even better. I came across an interesting article by Dan Holme (SharePoint MVP), where he collects some of the 2009 SharePoint predictions from fellow MOSS MVPs.

Meanwhile… we are continuing the series of posts on the “Myths and Truths of Email Management with SharePoint”. In his last post, Joel Oleson discussed some of the pifalls of using email enabled lists in SharePoint. I wanted to expand on that a little further.

While very powerful, email-enabled lists have some additional drawbacks that should be well understood before deploying them. These are described below:

1. When an email is sent to an email-enabled list, attachments are stripped off and stored separately.
2. The email body is stored as a .EML file, which cannot be opened in Outlook.
3. Users cannot specify different content types.
4. Users cannot specify custom metadata.

Colligo Contributor is an easy-to-deploy and manage .NET client Add-In for Outlook that enables users to move content to SharePoint through a simple drag-and-drop interface. It stores emails and attachments together in a single .MSG file, which can be opened in Outlook. Both content types and custom metadata can be set at the time of drag-and-drop. In addition, emails can be automatically moved to SharePoint using Outlook rules.

This is the fourth post in a guest series I’m doing here on Email Management in SharePoint. The third post was Myth #2.

Emailing a post to a blog … very cool or archiving an Exchange Discussion List to a SharePoint list … super cool … but be careful. Email-enabled self service lists can easily get out of control. Microsoft IT, which loves to use nearly every feature of SharePoint, decided against using email-enabled lists.

Email-enabled lists can be a significant IT resource drain. Without the proper planning and management, AD objects will be created with archiving and no lifecycle. Contact account naming standards are another reason. IT doesn’t want to see random contacts in AD.

Everyone wants to have the document library called “docs” and everyone wants to have the discussion list called “discussion”. If you have a process or even a workflow to get requests and manage these requests, you can better manage who needs them, when they are needed and for how long. So, my recommendation here is to know what you’re doing. Otherwise, it’s very easy to end up with a mess.

Like Public Folders, email-enabled lists can also pose security risks if not managed properly. Fortunately, they are more often secured to the context of the team so they are not as much of an exposure.

List scalability can pose problems with email-enabled lists. You don’t want to send all data from all users to one list. Put content in context in different site collections, sites, folders, as it relates to the context of the group, team, or project.

Most SharePoint environments don’t need the email-enabled functionality and the oversight it requires. Those that decide to use it should plan to set-up specific content objects and point the lists at them. Steve Smith’s document called “How to configure Incoming Email Enabled Libraries in MOSS2007 RTM using Exchange 2003 in an Active Directory Domain” explains in detail how to set-up email enabled lists. Don’t be surprised if it’s more complex to set-up than initially thought. I recommend setting it up in a preproduction environment first and learning how it works, then exercising administrative tasks and troubleshooting tasks around maintenance of the list, inbound SMTP, and AD contact objects.

In summary, I’d suggest you get answers to the following questions before implementing email-enabled SharePoint lists:

• Are those contact objects in a separate OU?
• How do you know if the inbound SMTP stops working?
• Can anyone send to the list?
• Does the item show up the way you expect it to, or are just attachments showing up?
• Does the metadata look like you expect it to?

The answers will help you better understand the nature of these special lists and help you better take advantage of this functionality, if you decide to use it.

Truth: Use with caution: To be successful, email-enabled lists require management & oversight to scale past a few thousand items.