SharePoint 2010 : Custom BCS connector for Search with Security Trimming, Batching, Incremental Crawling

This guide is designed to highlight the solution for enabling security trimming on search results in SharePoint 2010 for external database. It also focuses on large datasets and file system crawling. It provides an overview of the requirement, a solution for the same, and concludes with implementation of the solution suggested. In order to have item level security implemented and search result trimming based on security, we need to have the security descriptor generated for each record and present in the table for each row. BCS is used to pull data and index in SharePoint for implementing search driven applications, we need constant updation of data from the external system. In such systems we need incremental update to work to ensure CRUD operations. If you have documents stored in your External Systems then it is possible using the Business Connectivity Services (BCS) give your end users the ability to access and crawl these artifacts with ease. The Stream Accessor method allows you to pull System.Byte[] data from your External System and make it available from within SharePoint.

http://social.technet.microsoft.com/wiki/contents/articles/22945.sharepoint-2010-custom-bcs-connector-for-search-with-security-trimming-batching-incremental-crawling.aspx

How to programmatically update a content source in SharePoint 2010 enterprise / fast search

Following are the assembly references required to execute the code

using Microsoft.Office.Server.Search.Administration;
using Microsoft.SharePoint.Administration;
using Microsoft.SharePoint.BusinessData.Administration;
using Microsoft.SharePoint.BusinessData.Parser;
using Microsoft.SharePoint.BusinessData.SharedService;
using Microsoft.SharePoint.BusinessData.Infrastructure;
using Microsoft.BusinessData.Infrastructure;
using Microsoft.BusinessData.MetadataModel;
using Microsoft.Office.Server.Search;
using Microsoft.SharePoint.Search.Extended.Administration;
 

Here is the function that take two parameters as first is your central application url and other parameter to define if you want to configure fast search or enterprise search. You can use the code individually also in case you dont want to configure both.

public void UpdateContentSource(string WebAppUrl, bool IsFast)

        {

            try

            {

                string strContentSourceName = "<CustomContentSourceName>";

                string strLogSystemName = "<BDCLogSystemName>";

                //Content Source Update Started

                //execute this code if you are updaameting a fast search content source. else user for enterprise content source

                if (IsFast)

                {

                    foreach (SearchServiceApplication FASTServiceApp in SearchService.Service.SearchApplications)

                    {

                        SearchServiceApplicationType at = FASTServiceApp.SearchApplicationType;

                        if (FASTServiceApp.SearchApplicationType == SearchServiceApplicationType.ExtendedConnector)

                        {

                            Content FastContent = new Content(FASTServiceApp);

                            bool CheckExist = false;

                            ContentSourceCollection FastCSColl = FastContent.ContentSources;

                            foreach (ContentSource cs in FastCSColl)

                            {

                                if (cs.Name.Equals(strContentSourceName))

                                {

                                    CheckExist = true;

                                    Console.WriteLine(strContentSourceName + " FAST Search Content Source already exist");

                                }

                            }

                            BusinessDataContentSource CustomConSource = null;

                            if (CheckExist)

                            {

                                CustomConSource = (BusinessDataContentSource)FastCSColl[strContentSourceName];

                                if (CustomConSource.StartAddresses.Count > 0)

                                {

                                    CustomConSource.StartAddresses.Remove(BusinessDataContentSource.ConstructStartAddress("Default", new Guid("00000000-0000-0000-0000-000000000000"), strLogSystemName, strLogSystemName));

                                }

                                CustomConSource.Delete();

                                FastCSColl.Update();

                                Console.WriteLine(strContentSourceName + " FAST Search Content Source Deleted");

                            }

                            CustomConSource = (BusinessDataContentSource)FastCSColl.Create(typeof(BusinessDataContentSource), strContentSourceName);

                            CustomConSource.StartAddresses.Add(BusinessDataContentSource.ConstructStartAddress("Default", new Guid("00000000-0000-0000-0000-000000000000"), strLogSystemName, strLogSystemName));

                            CustomConSource.CrawlPriority = CrawlPriority.Normal;

                            FastCSColl.Update();

                            Console.WriteLine("External Source FAST Content Source Created");

                        }

                    }

                }

                else

                {

                    using (SPSite site = new SPSite(WebAppUrl))

                    {

                        foreach (SearchServiceApplication objServiceApp in SearchService.Service.SearchApplications)

                        {

                            if (objServiceApp.SearchApplicationType == SearchServiceApplicationType.Regular)

                            {

                                SearchServiceApplicationProxy proxy = (SearchServiceApplicationProxy)SearchServiceApplicationProxy.GetProxy(SPServiceContext.GetContext(site));

                                Guid appId = proxy.GetSearchServiceApplicationInfo().SearchServiceApplicationId;

                                SearchServiceApplication SearchApp = objServiceApp;         //SearchService.Service.SearchApplications.GetValue<SearchServiceApplication>(appId);

                                Content content = new Content(SearchApp);

                                ContentSourceCollection csColl = content.ContentSources;

                                bool CheckExist = false;

                                foreach (ContentSource cs in csColl)

                                {

                                    if (cs.Name.Equals(strContentSourceName))

                                    {

                                        CheckExist = true;

                                        Console.WriteLine(strContentSourceName + " is already exist");

                                    }

                                }

                                BusinessDataContentSource CustomConSource = null;

                                if (CheckExist)

                                {

                                    CustomConSource = (BusinessDataContentSource)csColl[strContentSourceName];

                                    if (CustomConSource.StartAddresses.Count > 0)

                                    {

                                        CustomConSource.StartAddresses.Remove(BusinessDataContentSource.ConstructStartAddress("Default", new Guid("00000000-0000-0000-0000-000000000000"), strLogSystemName, strLogSystemName));

                                    }

                                    CustomConSource.Delete();

                                    csColl.Update();

                                    Console.WriteLine(strContentSourceName + " Deleted");

                                }

                                CustomConSource = (BusinessDataContentSource)csColl.Create(typeof(BusinessDataContentSource), strContentSourceName);

                                CustomConSource.StartAddresses.Add(BusinessDataContentSource.ConstructStartAddress("Default", new Guid("00000000-0000-0000-0000-000000000000"), strLogSystemName, strLogSystemName));

                                CustomConSource.CrawlPriority = CrawlPriority.Normal;

                                csColl.Update();

                                Console.WriteLine("External Source Content Source Created");

                            }

                        }

                    }

                }

            }

            catch (Exception ex)

            {

                Console.WriteLine("Error:" + ex.Message);

            }

            Console.WriteLine("Content Source Update Ended");

        }

How to update BDC / BCS custom connector DLL programmatically in SharePoint 2010

using System.Management.Automation.Runspaces;

using System.Management.Automation;

public static void UpdateBDCDLL(string webappURL, string strModelName)

        {

            try

            {

// Mapping Assembly of BDC

                string DllPath = @"@"<DLL PHYSICAL PATH>";

                //create powershell runspace

                RunspaceConfiguration config = RunspaceConfiguration.Create();

                PSSnapInException OExSnapIn = null;

                PSSnapInInfo pssnap = config.AddPSSnapIn("Microsoft.SharePoint.PowerShell", out OExSnapIn);

                Runspace cmdlet = RunspaceFactory.CreateRunspace(config);                cmdlet.Open();

                RunspaceInvoke scriptInvoker = new RunspaceInvoke(cmdlet);

// set powershell execution policy to unrestricted

                scriptInvoker.Invoke("Set-ExecutionPolicy Unrestricted");  

// create a pipeline and load it with command object

                  Pipeline pipeline = cmdlet.CreatePipeline();

              StringBuilder sbPSCommand = new StringBuilder();

                sbPSCommand.AppendLine("$assemblyPath = '" + DllPath + "'");

                sbPSCommand.AppendLine("$lobSystem = '" + strModelName + "'");

                sbPSCommand.AppendLine("$url='" + webappURL + "'");

                sbPSCommand.AppendLine("$serviceContext = Get-SPServiceContext $url");

                sbPSCommand.AppendLine("$lobSystem = Get-SPBusinessDataCatalogMetadataObject -ServiceContext $serviceContext -BdcObjectType lobsystem -Name $lobSystem");

                sbPSCommand.AppendLine("Import-SPBusinessDataCatalogDotNetAssembly -LobSystem $lobSystem -Path $assemblyPath -Confirm:$false");

                Pipeline pipeline2 = cmdlet.CreatePipeline();

                pipeline2.Commands.AddScript(sbPSCommand.ToString());

                pipeline2.Invoke();

            }

            catch (Exception ex)

            {

                //exception handling goes here

            }

        }

Parameter count mismatch

If you are working with BCS to pull data for search service application or FAST Search then you might have encountered this error when ever you try to update a dll where you have added or removed parameters for either paging or some custom logic.

"Parameter Count Mismatch"

Here are the steps that you should use to resolve this issue :

1. Upload your dll in GAC
2. Run following commands in powershell



$url = "http://gdc-encore"
$assemblyPath = "Assembly path"
$lobSystem = "Lob system name"
Write-Host "Adding assembly to LOBSystem"
$serviceContext = Get-SPServiceContext $url
$lobSystem = Get-SPBusinessDataCatalogMetadataObject -ServiceContext $serviceContext -BdcObjectType lobsystem -Name $lobSystem
Import-SPBusinessDataCatalogDotNetAssembly -LobSystem $lobSystem -Path $assemblyPath



3. Reset search service from the services named "SharePoint Server Search 14" which is running mssdmn behind the scenes and caches the dll which causes the mismatch in assembly reference.