首页 Web开发 Windows程序 编程语言 数据库 移动开发 系统相关 微信 其他好文

[转]Hacking the iOS Spotlight

时间：2014-07-16 23:16:51   收藏：0   阅读：336

原文：http://theiostream.tumblr.com/post/36905860826/hacking-the-ios-spotlight

原文：http://theiostream.tumblr.com/post/54299348185/searchloader

原文：http://theiostream.tumblr.com/post/54339870761/hacking-the-ios-spotlight-v2

[Update] A lot of new findings have been uncovered! I will take the contents of this blog post into a decent iPhone Dev Wiki article as soon as I have the time.

It shall be noted that I might be wrong in how I approach some of these subjects. I have not experimented all possible cases for this topic, and I’ll edit the post as new findings come.

Back at the iOS 3.x betas,?KennyTM?createdCalculator.searchBundle, a way to show Calculator results on Spotlight at the iOS, just like in OSX!

Since then, many tweaks have attempted to improve Spotlight. Fromthis list:

I wanted to be able to add custom Spotlight results for user queries, just like Apple’s way of doing it, and after a week of disassembling, this is what I can tell:

The Hack

Spotlight gets its search results from?SPSearchQuery?which contain data such as the search string sent to?search bundles, which get a result from that query.

In both bundles, the?datastore?is the principal class, which will in some way give data for Spotlight to show. That data will be shown by SpringBoard.

Yet there are in fact many fundamental differences between those two:?Search Bundles?get the user query (serialized in theSPSearchQuery?class) through?searchd, and through the same mean they reply with a?SPSearchResult, directly parsed bySBSearchModel?in SpringBoard and displayed as UI.

Meanwhile,?Spotlight Bundles?have a complete different mean of functioning:?Extended.searchBundle?(a search bundle) uses theSPContentIndexer?class to look up a certain identified record store (some sort of special?AppSupport.framework?database) for data. From there they compose their?SPSearchResult?and send it over to Spotlight. Spotlight bundles are there for composing those record stores with data.

Search bundles also have to return a?-displayIdentifierForDomain:?method to associate the app for the search result with the search result (each result has a?domainfor it, which identifies it). The?-performQuery:withResultsPipe:?is used to create the search results from the passed-in query and send them over to searchd through the results pipe (SDActor?object).

Now, why don’t we create a search bundle to add results? Because every search query has an array of hardcoded domains (the existing Apple search bundles). And every search bundle’s array of domains (defined in?-searchDomains) must have one of its members as part of the search query’s domains.

OK, why not use one of the following existing domains?

Because then an exception will be thrown for two bundles adding results with the same domain!

I tried hooking?-[SPSearchQuery searchDomains]?and add a couple of different domains to it, but there was no success, since the same exception was thrown – even with my custom domain for both result and bundle.

With some more time, I might try reversing this system. Meanwhile, we shall move on.

KennyTM’s Calculator bundle was a search bundle which worked before Apple had this exception handling system, which allowed his bundle to run smoothly with an already hardcoded (but not fulfilled by any search bundle) domain.

That left me to hack into where no one had ever hacked before:Spotlight Bundles, and was successful. Now, I shall explain how does its system work in detail.

Spotlight directly iterates through all existing search bundles, reaching?Extended.searchBundle. Extended uses theSPContentIndexer?class to look over saved possible search results. Spotlight bundles fill this database with data.

Each spotlight bundle is registered inside/System/Library/Spotlight/domains.plist. The plist is an array of dictionaries which consist of the following main keys:

To get the Search Bundle, it’ll get the bundle for theSPDisplayIdentifier?and from its Info.plist get the following key values:

It is possible to have multiple categories for a same display identifier, generating double Spotlight results for that bundle.

With these registered, the following three files are created inside~/Library/Spotlight/<display identifier>:

To create a database entry, the following shall be done:

[[SPDaemonConnection sharedConnection] startRecordUpdatesForApplication:@"<display identifier>" andCategory:@"<category>];
[[SPDaemonConnection sharedConnection] requestRecordUpdatesForApplication:@"<display identifier>" category:@"<category>" andIDs:[NSArray arrayWithObject:@"<some external ID>"]];
[[SPDaemonConnection sharedConnection] endRecordUpdatesForApplication:@"<display identifier>" andCategory:@"<category>"];

With these updates to the?Updates file, the?AppIndexer?process is invoked and it deals with opening bundles and transferring their results to the record store.

[Update]?AppIndexer?will also be invoked using the Application Display ID as the ID passed into the?andIDs:?argument of?-requestRecordUpdatesForApplication:category:andIDs:if the record store is empty for every respring.

An example of sent ID is Messages.app’s: The Group ID for aCKConversation.

The?SPSpotlightDatastore?protocol is defined as follows:

A?NSDictionary *?shall be returned, parsing the ID into a full-fledged data holder for making up Search Results. In Messages.app’s, it’s a dictionary containing a whole conversation’s text as content, conversation recipients as title, etc.

Accepted keys are constants from Search.framework:

This returned?NSDictionary *?will complete the spotlight bundle cycle: Updates will be commited to the record store andExtended.searchBundle?will take care of showing all of those into Spotlight.

Now, clicking that result will launch the app. What then? How to show something specific inside it? I am completely clueless, yet I shall update this soon. I suspect?SPDaemonQueryDelegate?is the key for this problem (as present in?CKConversationSearcher?in ChatKit).

I guess that’s it. I’m working on a pretty cool tweak which will contain a bunch of these plugins, but will also release a MobileSubstrate extension to deal with the possible limitations of the Spotlight Bundle approach:

Credits on the finding also go to:

— Daniel Ferreira.

It’s been 7 months since I published my first article on this subject: Spotlight. Yes. That long. I will soon (that is, in a matter of hours) be writing a new technical writeup on the subject, named?Hacking the iOS Spotlight, v2.

Let me start by telling you a bit about how research was prior, and after the publishing of the article.

The whole thing started with a project I had with Ariel Aouizerate and Eliran Manzeli nicknamed Spotlight+, which aimed to extend Spotlight by displaying selected search results in a different way. This has not yet been implemented, even though it was the start of it all, and since the future of this project is yet unknown, I won’t disclose much else about it.

We soon realized that Spotlight lacked the categories of results we all wanted. Ariel thought we should display whatever we displayed through a?SearchCommands-like thing, but I figured, since we were doing this, we were going to do it right.

So the project changed its focus to a whole new direction. It was no longer about making Spotlight more practical, but about adding a whole new set of functionalities to it. Adding custom search results.

Then research started. I spent a week studying the SpringBoard, searchd and AppIndexer layers of it, and in a hurry released?Hacking the iOS Spotlight?in this blog.

The article covered an overview of existing software which extended Spotlight, a bit about the two kinds of search?plugins?–?Search Bundles?and the nicknamed?Spotlight Bundles, and how both functioned, and a small na?ve description of what domains were.

That writeup lacked two vital pieces of information, and had one error at its first release (which was then corrected with further research). It lacked:

Ever since, research has progressed into a complete piece of software, named?SearchLoader, which, as the name says, loads Search bundles and tricks Extended.searchBundle into loading our Spotlight bundles.

It was no simple task, but it will soon, as one would expect, be opensourced, with a through API documentation (in the upcoming technical article).

Loader started simply as a buggy loading thing, but it soon evolved into a stable Loader + Library set, with a lot of stuff one might need when developing their own search plugins. And when I see the first SearchLoader’s working code (which I had copied in my Mac) and compare it with the current code, I can barely believe it evolved so much.

With some short breaks from this project, I worked on some unfinished tweaks, did TweetAmplius, fixed stuff for iOS 6, wrote some of the Theos Docs, did some work on Oligos, re-beat the Mass Effect saga, played some Minecraft and Civilization, but now, it’s finished! I honestly can’t believe this!

To end this “say everything yet nothing” article, this wouldn’t have been possible without the tools of the trade:?class-dump, IDA,dyld_decache, and so forth.

And a huge thanks goes to my friends at the jailbreak community. Without them I would have most likely given up on this ;P.

So, expect cool stuff I’ve coded to come alongside Loader, and I’ll hopefully expect from you some cool plugins for SearchLoader. :)

Introduction

This article is purely technical. If you want to know a bit about the history of SearchLoader, go to?this blog post.

All of the data in this article reflects iOS 6. There has been an intermediate number of changes from iOS 5.

History:?The?TL?prefix for everything around SearchLoader and my projects related to Spotlight stands for?theiostream spotlight. It looked better than?SL, I could certainly not take?SP, and?TL?also looked nicer than?TP.

This article limits itself to the basic process of a search, and the creation of Search Bundles and Extended Domains. It does not go into detail on how each component of the Search framework work internally.

There are a couple of things which are irrelevant for this article, but some supposedly interesting areas of this subject might still make new articles. More precisely:

?

An overview of the search process

Spotlight is divided between two layers: SpringBoard (UI) and searchd (search).

SpringBoard has four?SBSearch...?classes:

SPSearchAgent, in SpringBoard, is incorporated bySBSearchModel‘s shared instance.

SBSearchController?asks for it to do what it’s meant to do: Take a query string, turn it into a?SPSearchQuery?and send it tosearchd.

The searchd layer uses?SPBundleManager?to load all existent search bundles (placed at?/System/Library/SearchBundlesor, with SearchLoader,/Library/SearchLoader/SearchBundles, and then it gets out of them a set of?datastore?objects.

Datastores on Search Bundles areNSObject<SPSearchDatastore> *?objects. These objects, through an API specified by the?SPSearchDatastore?protocol, perform a search through the query they receive and produce aSPSearchResult.

Search Results are sorted by an integer named a?search domain. Each search bundle provides a set of domains it “owns”.

When creating?SPSearchResults, internally or externally they will get placed inside a?SPSearchResultSection?object, which will be assigned to a domain.

Multiple sections can be added under the same domain, as done byApplication.searchBundle. The only search bundles to use multiple domains are?iPod.searchBundle?(due to unknown purposes) and?Extended.searchBundle?(each index gets one domain).

Back to SpringBoard, it gets sections for each domain and places them in the table view.

Yet, there is some special attention that should be paid toExtended.searchBundle?and Spotlight Bundles.

Extended.searchBundle?reads from?database or database-like entries?in some files to generate its results. The following content describes the generation of these entries. But as you can notice, they are?database entries. Therefore, this method of displaying search results should be used when results are not generated dynamically, but when they can be indexed.

SearchLoader edits?com.apple.search.appindexer.plist(the AppIndexer daemon’s?launchd.plist) so it’ll load MobileSubstrate. With this, it manages to control Spotlight Bundle loading.

Every time?searchd?is initialized (when Spotlight is brought up), it invokes the?AppIndexer?daemon.

On launch (usually), this daemon finds existing?Extended Domainsthrough?SPGetExtendedDomains(). This function reads from/System/Library/Spotlight/domains.plist?and returns an array of dictionaries. These dictionaries contain this domain’s display identifier (which reflects the generated search results refer to), a category (a string which usually has the format?<Name>Search) for it (a way to differentiate different search bundles/extended domains with the same display identifier due to referring to the same app) and required capabilities. This sort of dictionary is, therefore, namedextended domain.

Before going further, it’s important to introduce the file hierarchy for Spotlight Bundle databases, etc. Files related to extended domain with display identifier?com.yourcompany.test?and categoryTestSearch?will be placed at/var/mobile/Library/Spotlight/com.yourcompany.test. The files are:

From these extended domains, it usesSPDomainHasUpdatesFile()?to determine whether the updates file for an extended domain is empty. In case it is non-existent or contains updates, an?AppIndexer?instance is initialized with information about this extended domain.

Here, Spotlight Bundles (finally) get in the scene. They are loaded from?/System/Library/Spotlight/SearchBundles. Through a principal class of?NSObject<SPSpotlightDatastore> *?type, it generates a dictionary with specific keys which tells?AppIndexerhow it should index results into the content index/database, from anidentifier. If the updates file is empty, a list of?identifiers is created by the spotlight bundle itself. Else, the contents of the?updates file?are used. Therefore, it can be said that the updates file tracks identifiers that require indexing from the Spotlight bundle.

After getting this data,?AppIndexer?asks?searchd?to update the actual database/content index. Meanwhile, one might ask?Where do the identifiers for the update file come from?. The only native extended domain,?SMSSearch, uses a whole direct?ContentIndexwrapper to write to its update file (the?IMDSpotlight?function family from?IMCore). But happily, we don’t have to either useContentIndex, nor link to?IMCore. Apple provides some APIs inSPDaemonConnection, or even a whole framework just about that:Spotlight.framework?with?SPSpotlightManager.

And then we go back to the top.?Extended.searchBundle?will useSPContentIndexer?to look up contents of existent content indexes/databases and from them build Search Bundle-like results which will go to SpringBoard.

This finishes my Spotlight overview. The further sections will describe, respectively, the structure of a search bundle, an extended domain Spotlight Bundle (documenting?libspotlight?– a part of SearchLoader –’s APIs), how to make your bundle loadable by SearchLoader, some details about the SearchLoader tweak itself, SearchLoaderPreferences, and then will conclude.

Search Bundles

Search Bundles are composed of a principal class, which is theSearch Bundle datastore. It conforms to the?SPSearchDatastoreprotocol.

SearchLoader plugins actually should conform to theTLSearchDatastore?protocol, which conforms toSPSearchDatastore?and adds one method.

TLSearchDatastore

In this method, the search bundle should send its generatedSPSearchResult?or?SPSearchResultSection?objects back tosearchd?so it can be shown in the SpringBoard layer.

This method takes as arguments?query?and?pipe. They are the same (as of iOS 6:?SDSearchQuery?is a subclass ofSPSearchQuery), yet theoretically?query?should be used to obtain information regarding the search query (essentially, its query string, obtainable through the?- (NSString *)searchString;method), and?pipe?to send results back to?searchd, through the following methods:

In these methods, the?domain?argument should always be the search domain taken by the search bundle, the?section?parameter should be an initialized?SPSearchResultSection?to contain the desired search results, and?results?should be an array ofSPSearchResult?objects.

In case there is usage of the below-described?-(BOOL)blockDatastoreComplete?method and at some point asynchronous behavior happens, you should call?-[SDSearchQuery storeCompletedSearch:], passing?self?as a parameter, and?pipe?as an object.

It should return a?NSArray?object with?NSInteger?objects as its contents. Each?NSInteger?should hold an integer to serve as its taken search domain.

Due to a Loader limitation, in SearchLoader-loaded plugins only one search domain should be taken, else unknown results may be yielded.

This method should return a?NSString?object to represent the display identifier for a given domain. This display identifier is usually the application which search results reflect.

To perform some asynchronous-only tasks inside your search bundle or delegate-calling requesters, you can return?YES?on this method to block the?-[SDSearchQuery storeCompletedSearch:]method, therefore not progressing further in the search process and then rendering result committing from the datastore impossible.

Later, a call to?-[SDSearchQuery storeCompletedSearch:]should be placed as described above for the result to be actually committed, and obviously,?NO?should be returned here then for your call not to be subsequently blocked.

libspotlight APIs

The following?libspotlight?functions can be used for convenience or are required during the development ofSearchLoader-loaded search bundles:

(The internals of this function will be discussed further, and with it the need for a?category?parameter, which is characteristic of extended domains.)

This function gets the domain for a given display identifier (usually of the application which search results reflect) and a category string (defined above).

This?must?be the way to obtain the domain for a SearchLoader plugin, to avoid issues with other plugins.

This enables or disables the status bar activity indicator in SpringBoard. This should be used if you are loading content from the Internet.

This function is?completely unrelated?to the?-blockDatastoreComplete?method from?TLSearchDatastore.

Miscellaneous

A convention (made by me) states that you should?unless extremely required?never take over?3 seconds?with Internet requests.

Spotlight Bundles

SPSpotlightDatastore

From parameter?anId, a string which serves as an *identifier, this method should return a?NSDictionary?object with specific keys to represent a result. The?category?parameter is the extended domain’s category.

The following keys can have values assigned for in the returned dictionary. They are all constants defined in the SearchLoader headers, and part of?Spotlight.framework:

I should provide an image specifying which labels are which graphically soon. Meanwhile, you’ll have to experiment with it ;)

This should return an array of?NSString?objects to be passed into?-contentToIndexForID:inCategory:?as the?anId?parameter.

This method is called when the content index/database for given category is empty, and therefore it needs all existing data related to it put into identifiers, which will initially populate them.

An identifier has no proper definition nor standard, except the one that if it does not conform to URL standards it will not be put into the default-generated URL for it. The domain will be used instead. It can be as it best fits your parsing needs on?-contentToIndexForID:inCategory:. More details regarding default URL generation can be found below on the URL correction InfoBundle plist keys’ documentation.

To set a custom URL for a Spotlight bundle, the?TLCorrectURL...InfoBundle keys should be used. More details can be found below. This API is quite limited at the moment, but it can be expanded if a specific request regarding it is placed.

Updates File Manipulation

The following?SPSpotlightManager?method fromSpotlight.framework?can be used to modify the Updates file:

Obtains the shared instance for the?SPSpotlightManager?class.

This method adds the identifiers, described as?NSString?objects inside the?identifiers?parameter, to the updates file of extended domain of display identifier?displayID?and category?category.

Content Index/Database Manipulation

This method deletes the?/var/mobile/Library/Spotlightfiles for certain category of certain application for given display identifier.

This method triggers?AppIndexer, which will perform its on-launch tasks (update extended domains which require updating).

InfoBundles

InfoBundles are?document packages?(bundles without executables) placed inside?/Library/SearchLoader/Applications/. They tell SearchLoader which search/spotlight bundles placed at their respective directories should be loaded.

Required Keys

Required Keys for Search Bundles

Required Keys for Extended Domains

Optional Keys

The below keys show the process of creating your corrected URL with InfoBundle keys. It should be noted that the generated string should be a valid URL, else it will have no effect.

Correction works based on the manipulation of the original URL string. On search bundles, they are custom, and on Spotlight bundles, they take the following format:search://displayID/category/identifier.

It shall be noted that if?identifier?does not conform to URL standards, the original output URL after processing this string will have the?search://domain/record-entry-ID?format.

If there is no defined format and yet a delimiter, the default formatsearch://<$ID$>/<$C$>/%@?will be used.

Optional Keys for Extended Domains

SearchLoader

In the SpringBoard layer, SearchLoader changes:

In the searchd layer, the core hooks are:

These are the other hooks:

-[SPContentIndexer beginSearch:]: This is used to apply the restriction from the?TLQueryLengthMinimum?InfoBundle key.

-[SDSearchQuery storeCompletedSearch:]: Hooked to allow?TLSearchDatastore‘s?-blockDatastoreCompletemethod to be implemented.

On?AppIndexer, domain hooks are placed and the following:

Lastly, for the?TLRequireInternet(BOOL)?function fromlibspotlight?to work, a small Darwin notification system is placed inside the SpringBoard layer of the tweak, and when it receives a notification, it accordingly changes whether the status bar activity indicator is or not activated.

SearchLoaderPreferences

SearchLoader also hooks into Preferences to allow the native Spotlight preferences to know about our own plugins. It only applies the core hooks when?SearchSettings.bundle?is loaded.

Yet, it also adds a preference bundle of its own which thanks to rpetrich and DHowett’s?libprefs, can load – just like PreferenceLoader – preference entry plists! So you are allowed to – much like PreferenceLoader – place your plists exactly as you would with PL on?/Library/SearchLoader/Preferences. Neat, huh? :)

SearchLoader Limitations/Bugs

?

Conclusion

After 8 months of work in this area and some hours in this blog post (naturally not?only?on Loader, I’ve made my own share of Loader plugins to be released alongside it), I present this research. I hope it turns out to be useful. Seeing cool things being done with this is the best thing I could ever hope to achieve by making this.

I’d like to thank AAouiz and cydevlop for coordinating the Spotlight+ and SearchResults projects, which drove Loader to be created, and in no particular order Maximus, Cykey, DHowett, rms, fr0st, Nolan, ac3xx, his delightful wife, cj, Optimo, saurik and so many others who helped (directly or indirectly) to make this possible.

Finally, Loader has some fails, as the above section states, but I gladly take feature requests or bug reports.

[转]Hacking the iOS Spotlight,布布扣,bubuko.com

评论（0）
© 2014 mamicode.com 版权所有 京ICP备13008772号-2  联系我们:gaon5@hotmail.com
迷上了代码！