ceph对象存储折腾记_Python_第七城市
ceph对象存储折腾记
###前言一直想弄对象存储，以前弄过一次，不是很理解region是个什么东西，后来时间和工作上的原因没有再折腾，这两天闲了下来，再次折腾了一次。我是参考的ceph的中文翻译文挡进行的部署和测试。[传送门](http://docs./),文档里面介绍的和ceph本身的版本存在脱节的现象，可能初次接触的人会因为服务启动的问题摸不着头脑。###关于部署***ceph必要的软件包，配置好公共密钥和ceph mon的配置，这里我不再谈了。对象存储额外需要***的包是：ceph-radosgw和ceph-common***完毕你的系统上应该至少存在三个命令：rados、 radosgw 、 radosgw-admin其中整个对象网关服务就是由radosgw来启动的，radosgw-admin负责管理对象资源（用户，权限，bucket），rados基本算一个比较简单的s3客户端（？我这里可能理解不是很精确）####配置ceph.conf```[global]fsid = xxxxxxxxxxxxxxxxxxxxxxxxxxxxmon_initial_members = t41,t42,t45mon_host = 192.168.168.41,192.168.168.42,192.168.168.45auth_cluster_required = cephxauth_service_required = cephxauth_client_required = cephx# t56是服务器的hostname,由hostname -s命令可获取[client.radosgw.t56]host = your_ceph_rados_hostkeyring = /etc/ceph/ceph.client.radosgw.keyringrgw_socket_path = "/var/run/ceph/ceph.radosgw.gateway.fastcgi.sock"log_file = "/data/logs/client.radosgw.gateway.log"rgw_frontends = civetweb port=80rgw_print_continue = true```所有配置选项参考[传送门](http://docs./radosgw/config-ref/)####密钥环```#创建方式ceph auth create client.radosgw.t56 osd 'allow rwx' mon 'allow rwx' -o /etc/ceph/ceph.client.radosgw.keyring#密钥环的样子[client.radosgw.t56] key = xxxxxxxxxxxxxxxxxxxxxxxx==```####服务的启动方式```#有三种启动方式1. /etc/init.d/ceph-radosgw[start|stop|status|reload]2. systemctl startceph-radosgw3. radosgw-c /etc/ceph/ceph.conf -n client.radosgw.t56 ```本质都是第三种启动，无非写了个脚本而已。radosgw-h可以看看其他的参数，其中-f前台执行和--debug_ms设置调试等级有利于调试。####授权用户```[root@t56 /data]# radosgw-admin user create --uid="xueyi28" --display-name="Xueyi"{"user_id": "xueyi28","display_name": "Xueyi","email": "","suspended": 0,"max_buckets": 1000,"auid": 0,"subusers": [],"keys": [ { "user": "xueyi28", "access_key": "xxxxxxxxxxxxxxx", "secret_key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" }],"swift_keys": [],"caps": [],"op_mask": "read, write, delete","default_placement": "","placement_tags": [],"bucket_quota": { "enabled": false, "max_size_kb": -1, "max_objects": -1},"user_quota": { "enabled": false, "max_size_kb": -1, "max_objects": -1},"temp_url_keys": []}```- 这里有一个要注意的，这数据格式是json格式，access_key和secret_key上可能存在反斜杠之类的转意字符，用key的时候，注意把转意字符处理一下，省的纠结生成的key老是验证不过。####关于region对象存储设计考虑到数据中心区域的问题，这里的region就是区域的标识。比如中国西部数据中心，中国南方数据中心，不同的区域数据中心的bucket还可以异地同步（？下一步需要研究的），配合用户的DNS，可以让不同区域的用户连接不同区域的数据中心数据。```[root@t56 /data]# radosgw-admin region get{"name": "default","api_name": "","is_master": "true","endpoints": [],"hostnames": [],"master_zone": "","zones": [ { "name": "default", "endpoints": [], "log_meta": "false", "log_data": "false", "bucket_index_max_shards": 0 }],"placement_targets": [ { "name": "default-placement", "tags": [] }],"default_placement": "default-placement"}[root@t56 /data]#radosgw-admin regions list {"default_info": { "default_region": "default"},"regions": [ "default"]}```我这个是测试用的，就只有一个默认的default region。###数据的读写和bucket的使用我这里主要说php sdk使用s3接口。ceph给的文档里面的大多数sdk版本都是上个世纪的，亚马逊的s3 php sdk变的乱七八糟，入门比较慢，搞半天也搞不明白。我留了一个老版本的php sdk，凑合能接到ceph的文档。测试代码```&/?phpdefine('AWS_KEY', 'your_access_key');define('AWS_SECRET_KEY', 'your_secret_key');define('AWS_CANONICAL_ID', 'xueyi28');define('AWS_CANONICAL_NAME', 'Xueyi');$HOST = 'your_ceph_radosgw_host';// require the amazon sdk for php libraryrequire_once 'AWSSDKforPHP/sdk.class.php';// Instantiate the S3 class and point it at the desired host$Connection = new AmazonS3(array( 'key' => AWS_KEY, 'secret' => AWS_SECRET_KEY, 'canonical_id' => AWS_CANONICAL_ID, 'canonical_name' => AWS_CANONICAL_NAME,));$Connection->use_ssl = //禁用ssl$Connection->set_hostname($HOST);$Connection->enable_path_style(true); //采用path的模式，不然就是域名模式，bucket会成为根域名的子域名$Connection->allow_hostname_override(false);$Connection->path_style = //sdk哪里有点问题，path_style配置老是不生效，可以调一下sdk的代码#$Connection->create_bucket('my-new-bucket', AmazonS3::REGION_US_E1); //region可以在代码中进行定制，一个region对应一个地方域名。我们是default region 就对应REGION_US_E1，默认的就这个，看代码就明白了$ListResponse = $Connection->list_buckets();$Buckets = $ListResponse->body->Buckets->Bforeach ($Buckets as $Bucket) { echo $Bucket->Name . "/t" . $Bucket->CreationDate . "/n";}$Connection->create_object('my-new-bucket', 'hello.txt', array( 'body' => "Hello World!",));$Connection->set_object_acl('my-new-bucket', 'hello.txt', AmazonS3::ACL_PUBLIC);```php sdk[传送门](https://git.oschina.net/xueyi28/AWSSDKforPHP)核心逻辑代码/services/s3.class.php###终端玩转对象存储在终端下要用s3对象存储，最好的工具无非是[s3cmd](http://s3tools.org/s3cmd),一般yum就可以直接***下来，不过这玩意要注意版本，老版本的和最新的文档用法差别比较大。####首先要定制一下s3cmd的配置```[root@t56 ~]# s3cmd --configure Enter new values or accept defaults in brackets with Enter.Refer to user manual for detailed description of all options.Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.Access Key: xxxxxxxxxxxxxxxxxxxxxxxxSecret Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxDefault Region [US]: defaultEncryption password is used to protect your files from readingby unauthorized persons while in transfer to S3Encryption password: Path to GPG program [/usr/bin/gpg]: When using secure HTTPS protocol all communication with Amazon S3servers is protected from 3rd party eavesdropping. This method isslower than plain HTTP, and can only be proxied with Python 2.7 or newerUse HTTPS protocol [No]: NoOn some networks all internet access must go through a HTTP proxy.Try setting it here if you can't connect to S3 directlyHTTP Proxy server name: New settings:Access Key: xxxxxxxxxxxxxxxxxxxxxxxxxxSecret Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxDefault Region: defaultEncryption password: Path to GPG program: /usr/bin/gpgUse HTTPS protocol: FalseHTTP Proxy server name: HTTP Proxy server port: 0Test access with supplied credentials? [Y/n] nSave settings? [y/N] yConfiguration saved to '/root/.s3cfg'```定制完之后要再编辑一下/root/.s3cfg文件，把你自定义的Host写进去，定义好你的Host和region、bucket的访问方式给一个例子```[root@t56 ~]# cat .s3cfg [default]access_key = xxxxxxxxxxxxxxxxxxxxxxxaccess_token = add_encoding_exts = add_headers = bucket_location = defaultca_certs_file = cache_file = check_ssl_certificate = Truecloudfront_host = your_ceph_rgw_hostdefault_mime_type = binary/octet-streamdelay_updates = Falsedelete_after = Falsedelete_after_fetch = Falsedelete_removed = Falsedry_run = Falseenable_multipart = Trueencoding = ANSI_X3.4-1968encrypt = Falseexpiry_date = expiry_days = expiry_prefix = follow_symlinks = Falseforce = Falseget_continue = Falsegpg_command = /usr/bin/gpggpg_decrypt = %(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)sgpg_encrypt = %(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)sgpg_passphrase = guess_mime_type = Truehost_base = your_ceph_rgw_hosthost_bucket = your_ceph_rgw_host/%(bucket)human_readable_sizes = Falseignore_failed_copy = Falseinvalidate_default_index_on_cf = Falseinvalidate_default_index_root_on_cf = Trueinvalidate_on_cf = Falselist_md5 = Falselog_target_prefix = max_delete = -1mime_type = multipart_chunk_size_mb = 15preserve_attrs = Trueprogress_meter = Trueproxy_host = proxy_port = 0put_continue = Falserecursive = Falserecv_chunk = 4096reduced_redundancy = Falserestore_days = 1secret_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxsend_chunk = 4096server_side_encryption = Falsesignature_v2 = Falsesimpledb_host = sdb.your_ceph_rgw_hostskip_existing = Falsesocket_timeout = 300urlencoding_mode = normaluse_https = Falseuse_mime_magic = Trueverbosity = WARNINGwebsite_endpoint = http://%(bucket)s.s3-website-%(location)your_ceph_rgw_host/website_error = website_index = index.html```有几个地方修改你得按照你自己的需求去整，需要什么就整什么，让我们试试```[root@t56 ~]# s3cmd ls 12:36s3://my-new-bucket[root@t56 ~]# s3cmdls s3://my-new-bucket
12:38 12 s3://my-new-bucket/hello.txt[root@t56 ~]# s3cmd put test.logs3://my-new-bucket/test.logtest.log -> s3://my-new-bucket/test.log[1 of 1] 242 of 242 100% in0s 3.48 kB/sdone[root@t56 ~]# s3cmd setacl s3://my-new-bucket/test.log --acl-public s3://my-new-bucket/test.log: ACL set to Public[1 of 1][root@t56 ~]# s3cmd del s3://my-new-bucket/hello.txt File s3://my-new-bucket/hello.txt deleted//递归授权的方式 s3cmd setacl s3:/// --acl-public --recursive```棒极了
最新教程周点击榜
微信扫一扫